Thanks to @narthring the comment. I've learned a lot since the original post.

I've set the overall AFD timeout at 16 seconds, which is the minimum, which at least fails over if an app is spinning up after a restart. Our app takes around 90 seconds to respond properly, so when a recycle happens this will failover at least within 16 seconds.

We're using the HEAD request on a stripped back page to deal with the requests from the health monitor. With the settings as you confirmed above. Thanks for that.

I've managed to hook up devops with some powershell commands to manage blue/green deploys between the two origins, stopping and starting each origin between releases. This has achieve zero downtime with deploys which is a big tick.

I do think there could be a simpler offering, without the full CDN involved, but traffic manager doesn't work for our scenario. A Layer7 load balancer with more control over statuses returned to trigger failover.