My test shows: yes.
test before & after running (c:\usp\jobhp\ml_service_demo1\.conda) C:\usp\jobhp\ml_service_demo1>conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.1 -c pytorch -c nvidia
The increment of space taken:
24.9GB-16.2GB=8.7GB
The folder that takes the actual space:
C:\Users\nshln\miniconda3\pkgs - 8.87GB
The folder that appears to take up space, but actually not, it uses hardlink:
C:\usp\jobhp\ml_service_demo1 - 6.27GB
terminal output:
(c:\usp\jobhp\ml_service_demo1\.conda) C:\usp\jobhp\ml_service_demo1>conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.1 -c pytorch -c nvidia
Channels:
- pytorch
- nvidia
- defaults
Platform: win-64
Collecting package metadata (repodata.json): -
done
Solving environment: done
## Package Plan ##
environment location: c:\usp\jobhp\ml_service_demo1\.conda
added / updated specs:
- pytorch-cuda=12.1
- pytorch==2.5.1
- torchaudio==2.5.1
- torchvision==0.20.1
The following packages will be downloaded:
package | build
---------------------------|-----------------
blas-1.0 | mkl 6 KB
brotli-python-1.0.9 | py311h5da7b33_9 346 KB
certifi-2025.4.26 | py311haa95532_0 158 KB
charset-normalizer-3.3.2 | pyhd3eb1b0_0 44 KB
cuda-cccl-12.9.27 | 0 16 KB nvidia
cuda-cccl_win-64-12.9.27 | 0 1.1 MB nvidia
cuda-cudart-12.1.105 | 0 964 KB nvidia
cuda-cudart-dev-12.1.105 | 0 549 KB nvidia
cuda-cupti-12.1.105 | 0 11.6 MB nvidia
cuda-libraries-12.1.0 | 0 1 KB nvidia
cuda-libraries-dev-12.1.0 | 0 1 KB nvidia
cuda-nvrtc-12.1.105 | 0 73.2 MB nvidia
cuda-nvrtc-dev-12.1.105 | 0 16.5 MB nvidia
cuda-nvtx-12.1.105 | 0 41 KB nvidia
cuda-opencl-12.9.19 | 0 17 KB nvidia
cuda-opencl-dev-12.9.19 | 0 62 KB nvidia
cuda-profiler-api-12.9.19 | 0 19 KB nvidia
cuda-runtime-12.1.0 | 0 1 KB nvidia
cuda-version-12.9 | 3 17 KB nvidia
filelock-3.17.0 | py311haa95532_0 38 KB
freeglut-3.4.0 | hd77b12b_0 133 KB
freetype-2.13.3 | h0620614_0 554 KB
giflib-5.2.2 | h7edc060_0 105 KB
gmp-6.3.0 | h537511b_0 330 KB
gmpy2-2.2.1 | py311h827c3e9_0 205 KB
intel-openmp-2023.1.0 | h59b6b97_46320 2.7 MB
jinja2-3.1.6 | py311haa95532_0 359 KB
jpeg-9e | h827c3e9_3 334 KB
khronos-opencl-icd-loader-2024.05.08| h8cc25b3_0 100 KB
lcms2-2.16 | h62be587_1 568 KB
lerc-4.0.0 | h5da7b33_0 185 KB
libcublas-12.1.0.26 | 0 39 KB nvidia
libcublas-dev-12.1.0.26 | 0 348.3 MB nvidia
libcufft-11.0.2.4 | 0 6 KB nvidia
libcufft-dev-11.0.2.4 | 0 102.6 MB nvidia
libcurand-10.3.10.19 | 0 46.7 MB nvidia
libcurand-dev-10.3.10.19 | 0 242 KB nvidia
libcusolver-11.4.4.55 | 0 30 KB nvidia
libcusolver-dev-11.4.4.55 | 0 95.7 MB nvidia
libcusparse-12.0.2.55 | 0 12 KB nvidia
libcusparse-dev-12.0.2.55 | 0 162.5 MB nvidia
libdeflate-1.22 | h5bf469e_0 180 KB
libjpeg-turbo-2.0.0 | h196d8e1_0 618 KB
libnpp-12.0.2.50 | 0 305 KB nvidia
libnpp-dev-12.0.2.50 | 0 135.6 MB nvidia
libnvjitlink-12.1.105 | 0 67.3 MB nvidia
libnvjitlink-dev-12.1.105 | 0 13.8 MB nvidia
libnvjpeg-12.1.1.14 | 0 5 KB nvidia
libnvjpeg-dev-12.1.1.14 | 0 2.4 MB nvidia
libpng-1.6.39 | h8cc25b3_0 369 KB
libtiff-4.7.0 | h404307b_0 1.1 MB
libuv-1.48.0 | h827c3e9_0 322 KB
libwebp-1.3.2 | h18467be_1 83 KB
libwebp-base-1.3.2 | h3d04722_1 303 KB
lz4-c-1.9.4 | h2bbff1b_1 152 KB
markupsafe-3.0.2 | py311h827c3e9_0 39 KB
mkl-2023.1.0 | h6b88ed4_46358 155.9 MB
mkl-service-2.4.0 | py311h827c3e9_2 67 KB
mkl_fft-1.3.11 | py311h827c3e9_0 176 KB
mkl_random-1.2.8 | py311hea22821_0 266 KB
mpc-1.3.1 | h827c3e9_0 85 KB
mpfr-4.2.1 | h56c3642_0 266 KB
mpmath-1.3.0 | py311haa95532_0 1.0 MB
networkx-3.4.2 | py311haa95532_0 3.1 MB
numpy-2.0.1 | py311hdab7c0b_1 11 KB
numpy-base-2.0.1 | py311hd01c5d8_1 9.6 MB
openjpeg-2.5.2 | h9b5d1b5_1 268 KB
pillow-11.1.0 | py311hea0d53e_1 916 KB
pysocks-1.7.1 | py311haa95532_0 36 KB
pytorch-2.5.1 |py3.11_cuda12.1_cudnn9_0 1.21 GB pytorch
pytorch-cuda-12.1 | hde6ce7c_6 7 KB pytorch
pytorch-mutex-1.0 | cuda 3 KB pytorch
pyyaml-6.0.2 | py311h827c3e9_0 205 KB
requests-2.32.3 | py311haa95532_1 128 KB
sympy-1.13.3 | py311haa95532_1 15.5 MB
tbb-2021.8.0 | h59b6b97_0 149 KB
torchaudio-2.5.1 | py311_cu121 7.2 MB pytorch
torchvision-0.20.1 | py311_cu121 7.9 MB pytorch
urllib3-2.3.0 | py311haa95532_0 242 KB
win_inet_pton-1.1.0 | py311haa95532_0 10 KB
yaml-0.2.5 | he774522_0 62 KB
zstd-1.5.6 | h8880b57_0 708 KB
------------------------------------------------------------
Total: 2.47 GB
The following NEW packages will be INSTALLED:
blas pkgs/main/win-64::blas-1.0-mkl
brotli-python pkgs/main/win-64::brotli-python-1.0.9-py311h5da7b33_9
certifi pkgs/main/win-64::certifi-2025.4.26-py311haa95532_0
charset-normalizer pkgs/main/noarch::charset-normalizer-3.3.2-pyhd3eb1b0_0
cuda-cccl nvidia/win-64::cuda-cccl-12.9.27-0
cuda-cccl_win-64 nvidia/win-64::cuda-cccl_win-64-12.9.27-0
cuda-cudart nvidia/win-64::cuda-cudart-12.1.105-0
cuda-cudart-dev nvidia/win-64::cuda-cudart-dev-12.1.105-0
cuda-cupti nvidia/win-64::cuda-cupti-12.1.105-0
cuda-libraries nvidia/win-64::cuda-libraries-12.1.0-0
cuda-libraries-dev nvidia/win-64::cuda-libraries-dev-12.1.0-0
cuda-nvrtc nvidia/win-64::cuda-nvrtc-12.1.105-0
cuda-nvrtc-dev nvidia/win-64::cuda-nvrtc-dev-12.1.105-0
cuda-nvtx nvidia/win-64::cuda-nvtx-12.1.105-0
cuda-opencl nvidia/win-64::cuda-opencl-12.9.19-0
cuda-opencl-dev nvidia/win-64::cuda-opencl-dev-12.9.19-0
cuda-profiler-api nvidia/win-64::cuda-profiler-api-12.9.19-0
cuda-runtime nvidia/win-64::cuda-runtime-12.1.0-0
cuda-version nvidia/noarch::cuda-version-12.9-3
filelock pkgs/main/win-64::filelock-3.17.0-py311haa95532_0
freeglut pkgs/main/win-64::freeglut-3.4.0-hd77b12b_0
freetype pkgs/main/win-64::freetype-2.13.3-h0620614_0
giflib pkgs/main/win-64::giflib-5.2.2-h7edc060_0
gmp pkgs/main/win-64::gmp-6.3.0-h537511b_0
gmpy2 pkgs/main/win-64::gmpy2-2.2.1-py311h827c3e9_0
intel-openmp pkgs/main/win-64::intel-openmp-2023.1.0-h59b6b97_46320
jinja2 pkgs/main/win-64::jinja2-3.1.6-py311haa95532_0
jpeg pkgs/main/win-64::jpeg-9e-h827c3e9_3
khronos-opencl-ic~ pkgs/main/win-64::khronos-opencl-icd-loader-2024.05.08-h8cc25b3_0
lcms2 pkgs/main/win-64::lcms2-2.16-h62be587_1
lerc pkgs/main/win-64::lerc-4.0.0-h5da7b33_0
libcublas nvidia/win-64::libcublas-12.1.0.26-0
libcublas-dev nvidia/win-64::libcublas-dev-12.1.0.26-0
libcufft nvidia/win-64::libcufft-11.0.2.4-0
libcufft-dev nvidia/win-64::libcufft-dev-11.0.2.4-0
libcurand nvidia/win-64::libcurand-10.3.10.19-0
libcurand-dev nvidia/win-64::libcurand-dev-10.3.10.19-0
libcusolver nvidia/win-64::libcusolver-11.4.4.55-0
libcusolver-dev nvidia/win-64::libcusolver-dev-11.4.4.55-0
libcusparse nvidia/win-64::libcusparse-12.0.2.55-0
libcusparse-dev nvidia/win-64::libcusparse-dev-12.0.2.55-0
libdeflate pkgs/main/win-64::libdeflate-1.22-h5bf469e_0
libjpeg-turbo pkgs/main/win-64::libjpeg-turbo-2.0.0-h196d8e1_0
libnpp nvidia/win-64::libnpp-12.0.2.50-0
libnpp-dev nvidia/win-64::libnpp-dev-12.0.2.50-0
libnvjitlink nvidia/win-64::libnvjitlink-12.1.105-0
libnvjitlink-dev nvidia/win-64::libnvjitlink-dev-12.1.105-0
libnvjpeg nvidia/win-64::libnvjpeg-12.1.1.14-0
libnvjpeg-dev nvidia/win-64::libnvjpeg-dev-12.1.1.14-0
libpng pkgs/main/win-64::libpng-1.6.39-h8cc25b3_0
libtiff pkgs/main/win-64::libtiff-4.7.0-h404307b_0
libuv pkgs/main/win-64::libuv-1.48.0-h827c3e9_0
libwebp pkgs/main/win-64::libwebp-1.3.2-h18467be_1
libwebp-base pkgs/main/win-64::libwebp-base-1.3.2-h3d04722_1
lz4-c pkgs/main/win-64::lz4-c-1.9.4-h2bbff1b_1
markupsafe pkgs/main/win-64::markupsafe-3.0.2-py311h827c3e9_0
mkl pkgs/main/win-64::mkl-2023.1.0-h6b88ed4_46358
mkl-service pkgs/main/win-64::mkl-service-2.4.0-py311h827c3e9_2
mkl_fft pkgs/main/win-64::mkl_fft-1.3.11-py311h827c3e9_0
mkl_random pkgs/main/win-64::mkl_random-1.2.8-py311hea22821_0
mpc pkgs/main/win-64::mpc-1.3.1-h827c3e9_0
mpfr pkgs/main/win-64::mpfr-4.2.1-h56c3642_0
mpmath pkgs/main/win-64::mpmath-1.3.0-py311haa95532_0
networkx pkgs/main/win-64::networkx-3.4.2-py311haa95532_0
numpy pkgs/main/win-64::numpy-2.0.1-py311hdab7c0b_1
numpy-base pkgs/main/win-64::numpy-base-2.0.1-py311hd01c5d8_1
openjpeg pkgs/main/win-64::openjpeg-2.5.2-h9b5d1b5_1
pillow pkgs/main/win-64::pillow-11.1.0-py311hea0d53e_1
pysocks pkgs/main/win-64::pysocks-1.7.1-py311haa95532_0
pytorch pytorch/win-64::pytorch-2.5.1-py3.11_cuda12.1_cudnn9_0
pytorch-cuda pytorch/win-64::pytorch-cuda-12.1-hde6ce7c_6
pytorch-mutex pytorch/noarch::pytorch-mutex-1.0-cuda
pyyaml pkgs/main/win-64::pyyaml-6.0.2-py311h827c3e9_0
requests pkgs/main/win-64::requests-2.32.3-py311haa95532_1
sympy pkgs/main/win-64::sympy-1.13.3-py311haa95532_1
tbb pkgs/main/win-64::tbb-2021.8.0-h59b6b97_0
torchaudio pytorch/win-64::torchaudio-2.5.1-py311_cu121
torchvision pytorch/win-64::torchvision-0.20.1-py311_cu121
urllib3 pkgs/main/win-64::urllib3-2.3.0-py311haa95532_0
win_inet_pton pkgs/main/win-64::win_inet_pton-1.1.0-py311haa95532_0
yaml pkgs/main/win-64::yaml-0.2.5-he774522_0
zstd pkgs/main/win-64::zstd-1.5.6-h8880b57_0
Proceed ([y]/n)? y
find the hard link of one file by fsutil hardlink list <your_path>
nsht@AZIH C:\Users\nshln
$ fsutil hardlink list "C:/usp/jobhp/ml_service_demo1/.conda/Lib/site-packages/torch/lib/torch_cuda.dll"
\Users\nshln\miniconda3\pkgs\pytorch-2.5.1-py3.11_cuda12.1_cudnn9_0\Lib\site-packages\torch\lib\torch_cuda.dll
\usp\jobhp\ml_service_demo1\.conda\Lib\site-packages\torch\lib\torch_cuda.dll
not important:
There is a related topic for a javascript node_modules
package manager pnpm
:
javascript
~= python
pnpm
~= conda
npm
~= pip
node_modules
~= .venv
/ .conda
Why does my
node_modules
folder use disk space if packages are stored in a global store?pnpm creates hard links from the global store to the project's
node_modules
folders. Hard links point to the same place on the disk where the original files are. So, for example, if you havefoo
in your project as a dependency and it occupies 1MB of space, then it will look like it occupies 1MB of space in the project'snode_modules
folder and the same amount of space in the global store. However, that 1MB is the same space on the disk addressed from two different locations. So in totalfoo
occupies 1MB, not 2MB.For more on this subject:
- Why do hard links seem to take the same space as the originals?
- A thread from the pnpm chat room
- An issue in the pnpm repo
Does it work on Windows?
Short answer: Yes. Long answer: Using symbolic linking on Windows is problematic to say the least, however, pnpm has a workaround. For Windows, we use junctions instead.
more related:
pnpm like installation mode to save space · Issue #3487 · python-poetry/poetry
https://github.com/python-poetry/poetry/issues/3487
python - Does anaconda support hardlink in Windows? - Stack Overflow
https://stackoverflow.com/questions/75657317/does-anaconda-support-hardlink-in-windows
pnpm for Python · pnpm · Discussion #3164
https://github.com/orgs/pnpm/discussions/3164
- $>"
- **Use conda environments for isolation**
- - Create a conda environment to isolate any changes pip makes.
- Environments take up little space thanks to hard links.
<$
https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html
- $>"
There’s not a whole lot to this. [Conda tries to use hard links whenever possible](https://github.com/conda/conda/blob/4.6.7/conda/gateways/disk/create.py#L297-L337). Hard links increase the reference count on disk to a specific file, without actually copying its contents.
<$
https://www.anaconda.com/blog/understanding-and-improving-condas-performance
pnpm, but for Python? - DEV Community
https://dev.to/ghost/pnpm-but-for-python-3hb3