As per C3roe's comment, the issue in fact was with the size of the label field itself being much higher than the rendered SVG it contains.

This was solved very simply by adding height and width attributes to the SVG tag itself, the label then only surrounded the SVG element.