cross posting the answer from the uv issue tracker

This can be solved by explicitely telling uv it cannot install torch+cu124 on aarch64 by adding a marker in the dependencies:

cu124 = [
    "torch==2.6.0+cu124; sys_platform == 'linux' and platform_machine == 'x86_64'", 
]