It turned out that the problem is not in the authorization of the GraphQL-Backend but rather in GraphQL-UI. If you connect with a regular client (like https://github.com/graphql-python/gql/blob/master/docs/code_examples/websockets_async.py) authentication works as expected.
Issue in the SmallRye GraphQL-UI is tracked at https://github.com/smallrye/smallrye-graphql/issues/2255