Turns out I was indeed using the SDK wrong. I was using a custom user object and custom methods to log in and then you manually need to provide the header indeed...
Next time just use the provided methods.