The solution might be to customize RestartClassLoader with META-INF/spring-devtools.properties and the definitions of restart.exclude & restart.include:
https://docs.spring.io/spring-boot/reference/using/devtools.html#using.devtools.restart.customizing-the-classload