Turns out the 2.8.6 version of swagger-ui doesn't work with SpringBoot 3.4.4 at the moment. Downgrading to 2.8.5 worked just fine for me.
<dependency>
<groupId>org.springdoc</groupId>
<artifactId>springdoc-openapi-starter-webmvc-ui</artifactId>
<version>2.8.5</version>
</dependency>