The filter is not the responsible ! It was a Keycloak configuration issue.
When you give the uma_protection role of a client to a user, the roles client scope automatically adds the client to the audience of access token through the audience resolve mapper.
To test the authorization I was removing the uma_protection role of the client from the user. Thus, the client was not added to the audience of the access token anymore.
The goal is to add the client to the audience of the access token.
Don't know if other options are available, but the one that we chose is to create a client scope with a token mapper of type Audience having the client included :
We then add this scope to each client that needs an access token allowed to request the first one: