How incredibly frustrating... I have found a workaround.

I have updated my endpoint to take in a base64 encoded string.

And I have updated to Power Automate flow to use the Get Attachment (V2) call instead of using the New email trigger's attachments.

For some reason it works now... I have no idea what the different between the attachments from the trigger and the attachments from the call are, but switching it has worked.

And I'm fairly confident the problem was with getting the attachments from the trigger, and not the Get Attachment (V2) call