I had same issue with demucs which uses pytorch --- I needed to install both `torchaudio` and `soundfile` in conda isolated environment
pip install torchaudio soundfile