Beyond handling nil data recording with timeouts, there are several additional considerations to optimize your system:
Bloom Filter Implementation: You could apply a Bloom filter at the API endpoint for internal data queries. This would accelerate query processing while reducing unnecessary workload by efficiently identifying non-existent entries before executing expensive operations.
Singleflight Pattern: Consider adopting the singleflight pattern (as implemented in Golang's https://pkg.go.dev/golang.org/x/sync/singleflight). This becomes particularly helpful when cached nil data expires and multiple concurrent requests arrive simultaneously. Without an existing cache entry, these parallel requests would bypass the cache layer and directly hit the upstream API endpoint, potentially causing request stampeding that could overwhelm backend systems (also known as cache penetration).