This might be a late, but I would like to answer based on our experience, since we had exactly the same issue.
In this case, it's not so much the template which causes the slowdown, but the function itself.
Even though you put inline, there is still some overhead with jumps, extra variables and locality issues compared with the macro.
In our simulation solver we also got more than 10% improvement by changing the identical code from an inline function to equivalent macro.
We were really surprised, because we expected that compiler would optimize better.
I guess it's mostly fine for general purpose, but when you really have to squeeze all the performance, simpler (but uglier) is better.