In the Spring docs it is written

https://docs.spring.io/spring-framework/reference/web/webflux/controller/ann-advice.html

that @ControllerAdvice annotation can be made more specific by annotation attributes. In my case, the rest controllers in the system are annotated with @RestController so I can narrow down the scope of my controller advice to them with this:

@ControllerAdvice(annotations = RestController.class)
@Order(Ordered.HIGHEST_PRECEDENCE)
public class RestExceptionHandler extends ResponseEntityExceptionHandler {
...
}