After some more research, I found that mps is only built in the pytorch nightly release, which is installable via:
conda install pytorch-nightly::pytorch -c pytorch-nightly
https://pytorch.org/get-started/locally/