As of December 2024, this is also possible through the Azure portal. Simply navigate to your container app and click on Stop on the top:
Stop
Enjoy!