It turns out that only old version of Alert Triggers includes the top 10 results. It took a bit of effort to prove it out, but it is possible to deploy an Alert Trigger using the 2018-06 api through the portal - the Top 10 Results section was present. In our case our Terraform scripts were also targeting the legacy api, which added to the confusion. I wouldn't recommend to use the legacy api though...