It seems this was user error (kindof). In figma I have a layer that is used as a mask (to clip the svg). Every other program doesnt have an issue with it having a color, but Godot 4.4 seems to apply it anyway.

Here is my figma file:

as you can see, the Mask has a color and a blend mode, and that gets applied in Godot, making it darker. If I set the "Fill" to white, everything looks ok.