After spending considerable time investigating this issue, I was unable to pinpoint the exact cause of the random data loss in Redis running on Azure Container Instances (ACI), even after trying different configurations of redis.conf. The problem persisted, and I couldn't achieve the stability needed for my use case.

To address this and make progress, I decided to deploy Redis on an Azure Virtual Machine (VM) within a private VNet (using terraform). This approach proved to be stable, and I was able to maintain a reliable Redis database. I chose this solution for two main reasons:

1. Cost Efficiency: Running Redis on an Azure VM allowed better cost control compared to other managed solutions.
2. Flexibility: It gave me full control to configure and manage the infrastructure according to my needs.

That said, for scenarios where quick deployment and minimal development effort are priorities, Azure Redis Cache could be a viable alternative. It offers a managed Redis service with built-in reliability and persistence, making it ideal for use cases that require minimal infrastructure management.

This solution might help others facing similar issues or looking for a balance between cost, control, and ease of use.