I hit this issue as well. Wow does memory usage go out of control with 24 threads (Ryzen 5900X). Looking in the setup.py file for Triton, I saw that it uses MAX_JOBS on Linux, so, in the Triton source directory:

MAX_JOBS="6" pip3 install .

That keeps the maximum number of threads to 6. It seems each thread will use 1-2.5GB of RAM, so for 32 threads which I see in your question, you'd need at least 80Gb of RAM.

I was following instructions here when I hit this issue: https://docs.vllm.ai/en/latest/getting_started/amd-installation.html

Modify your step #6 to include MAX_JOBS="[maximum number of parallel jobs]", which I think with 48GB of RAM, not to go past 16. Maybe try 8 to be conservative.