If your application is still predicting when it receives the SIGTERM, it means the container is being terminated before it can complete the prediction.

To handle this, you can implement a SIGTERM handler in your application. When the handler receives the SIGTERM, it should start failing the readiness probe. This will prevent new requests from being routed to the container, and allow it to finish processing the current request before shutting down.

For writing a component to cancel the underlying resources you may visit this public documentation which includes sample code that shows how to attach a SIGTERM handler.