I generally front-load the build into the CI pipeline and publish a container image that already contains the compiled binary. Then the pods simply pull that image and execute in seconds. If I still need incremental builds inside Kubernetes, I can mount a network-backed cache like EFS for read-only dependency caches.