I discovered that I need to disable the alert in order to change the threshold.

To change a project-level MongoDB Alert:

1. Sign in to MongoDB Atlas.
2. Go to the Projects page for the desired Organization.
3. Click on the project that has the alert to be modified. In the upper-left hand portion of the screen there is a project drop-down box.
4. Click the kabob icon (three vertical dots) to the right of the project drop-down box and select "Project Settings". The Project Settings page will appear with a navigation pane on the left.
5. Click "Alerts" in the navigation pane.
6. Disable the desired alert.
7. Edit it and set it to the desired configuration.
8. Enable the alert.