I had to upgrade setuptools and wheel:

C:/Program Files/ComfyUI/ComfyUI_windows_portable/python_embeded/python.exe -m pip install -U pip setuptools wheel

Then I could install xformers.