In production, it's always best to be prepared for the worst. Unexpected failures can happen at any time for many reasons. Azure Container Apps are generally stable, but things can still go wrong—especially during upgrades or system changes.

For example, I recently faced an issue with my cluster. I upgraded the master node and three worker nodes without any problems. But when I started working on the fourth node, something went wrong. The drain command didn’t work, and all my pods got stuck in a terminating state.

I followed the same steps as before, but for some reason, this time it failed. This showed me that even when everything seems fine, unexpected issues can happen.

That’s why it’s important to plan for failures. If we prepare for things like forced shutdowns, crashes, or node failures, we can save time and effort and keep our system running smoothly. Does not matter if this happen often or not but you should be ready for it.