Recently came across this situation.

Query is as below:

traces | where timestamp between (ago(10m) .. ago(5m))

Aggregation granularity = 5 minutes; Frequency of evaluation = 5 minutes; In the advanced options, Number of violations = 1; Evaluation period = 5 minutes

Explanation: if alert runs at 06:00 AM, then query will collect data between 05:50 AM to 05:55 AM. As Aggregation granularity set to 5minutes, 1 data point will be produced. It will be checked against the threshold and alert triggered based on the condition met/not. Here considered 5minutes buffer for azure resources to send their logs to Azure Monitor tables.

Choose Aggregation granularity, Frequency of evaluation, Evaluation period, Number of violations as per the requirement.