There is a property that enables it for all tests.

spring.test.observability.auto-configure=true

I found this property in the source code of ObservabilityContextCustomizerFactory, which is part of spring-boot-test-autoconfigure-3.1.1.jar