On further debugging, I realized that the SecurityFilterChain bean was not recognized, hence the matcher was never invoked.

I was able to make it work with the following changes:

spring-cloud.version=2024.0.0 with spring-boot-starter-webflux

Using @EnableWebFluxSecurity instead of @EnableWebSecurity and setting up the filter chain as

@Bean
    public SecurityWebFilterChain securityFilterChain(ServerHttpSecurity http){...}