I managed to fix it by utilizing onTokenValidated and injecting custom claims into the context. Works perfectly with the Authorize attribute.
onTokenValidated
Authorize