conda-pack
conda-pack copied to clipboard
conda
permission error in conda-unpack on windows 10
Checklist
- [X] I added a descriptive title
- [X] I searched open reports and couldn't find a duplicate
What happened?
I'm moving an env in an attempt to create an offline system backup or system transfer. Issue is occurring when trying to transfer it to a test unit. Please see the attached screenshot.
Conda Info
active environment : base
active env location : C:\Users\vicko\anaconda3
shell level : 1
user config file : C:\Users\vicko\.condarc
populated config files : C:\Users\vicko\.condarc
conda version : 22.9.0
conda-build version : 3.22.0
python version : 3.9.13.final.0
virtual packages : __cuda=11.6=0
__win=0=0
__archspec=1=x86_64
base environment : C:\Users\vicko\anaconda3 (writable)
conda av data dir : C:\Users\vicko\anaconda3\etc\conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/win-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/win-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/msys2/win-64
https://repo.anaconda.com/pkgs/msys2/noarch
package cache : C:\Users\vicko\anaconda3\pkgs
C:\Users\vicko\.conda\pkgs
C:\Users\vicko\AppData\Local\conda\conda\pkgs
envs directories : C:\Users\vicko\anaconda3\envs
C:\Users\vicko\.conda\envs
C:\Users\vicko\AppData\Local\conda\conda\envs
platform : win-64
user-agent : conda/22.9.0 requests/2.28.1 CPython/3.9.13 Windows/10 Windows/10.0.19044
administrator : False
netrc file : None
offline mode : False
Conda Config
==> C:\Users\vicko\.condarc <==
channels:
- defaults
Conda list
# packages in environment at C:\Users\vicko\anaconda3\envs\wt_:
#
# Name Version Build Channel
blas 1.0 mkl defaults
brotlipy 0.7.0 py39h2bbff1b_1003 defaults
ca-certificates 2022.07.19 haa95532_0 defaults
certifi 2022.9.24 py39haa95532_0 defaults
cffi 1.15.1 py39h2bbff1b_0 defaults
charset-normalizer 2.0.4 pyhd3eb1b0_0 defaults
colorama 0.4.6 pypi_0 pypi
cryptography 37.0.1 py39h21b164f_0 defaults
cudatoolkit 11.3.1 h59b6b97_2 defaults
ffmpeg 1.4 pypi_0 pypi
ffmpeg-python 0.2.0 pypi_0 pypi
filelock 3.8.0 pypi_0 pypi
freetype 2.10.4 hd328e21_0 defaults
future 0.18.2 py39haa95532_1 defaults
huggingface-hub 0.10.1 pypi_0 pypi
idna 3.4 py39haa95532_0 defaults
intel-openmp 2021.4.0 haa95532_3556 defaults
jpeg 9e h2bbff1b_0 defaults
lerc 3.0 hd77b12b_0 defaults
libdeflate 1.8 h2bbff1b_5 defaults
libpng 1.6.37 h2a8f88b_0 defaults
libtiff 4.4.0 h8a3f274_0 defaults
libuv 1.40.0 he774522_0 defaults
libwebp 1.2.4 h2bbff1b_0 defaults
libwebp-base 1.2.4 h2bbff1b_0 defaults
lz4-c 1.9.3 h2bbff1b_1 defaults
mkl 2021.4.0 haa95532_640 defaults
mkl-service 2.4.0 py39h2bbff1b_0 defaults
mkl_fft 1.3.1 py39h277e83a_0 defaults
mkl_random 1.2.2 py39hf11a4ad_0 defaults
more-itertools 9.0.0 pypi_0 pypi
ninja 1.10.2 haa95532_5 defaults
ninja-base 1.10.2 h6d14046_5 defaults
numpy 1.23.3 py39h3b20f71_0 defaults
numpy-base 1.23.3 py39h4da318b_0 defaults
openssl 1.1.1q h2bbff1b_0 defaults
packaging 21.3 pypi_0 pypi
pillow 9.2.0 py39hdc2b20a_1 defaults
pip 22.2.2 py39haa95532_0 defaults
pycparser 2.21 pyhd3eb1b0_0 defaults
pyopenssl 22.0.0 pyhd3eb1b0_0 defaults
pyparsing 3.0.9 pypi_0 pypi
pysocks 1.7.1 py39haa95532_0 defaults
python 3.9.13 h6244533_2 defaults
pytorch 1.12.1 py3.9_cuda11.3_cudnn8_0 pytorch
pytorch-mutex 1.0 cuda pytorch
pyyaml 6.0 pypi_0 pypi
regex 2022.9.13 pypi_0 pypi
requests 2.28.1 py39haa95532_0 defaults
setuptools 63.4.1 py39haa95532_0 defaults
six 1.16.0 pyhd3eb1b0_1 defaults
sqlite 3.39.3 h2bbff1b_0 defaults
tk 8.6.12 h2bbff1b_0 defaults
tokenizers 0.13.1 pypi_0 pypi
torchaudio 0.12.1 py39_cu113 pytorch
torchvision 0.13.1 py39_cu113 pytorch
tqdm 4.64.1 pypi_0 pypi
transformers 4.23.1 pypi_0 pypi
typing-extensions 4.3.0 py39haa95532_0 defaults
typing_extensions 4.3.0 py39haa95532_0 defaults
tzdata 2022e h04d1e81_0 defaults
urllib3 1.26.12 py39haa95532_0 defaults
vc 14.2 h21ff451_1 defaults
vs2015_runtime 14.27.29016 h5e58377_2 defaults
wheel 0.37.1 pyhd3eb1b0_0 defaults
whisper 1.0 pypi_0 pypi
win_inet_pton 1.1.0 py39haa95532_0 defaults
wincertstore 0.2 py39haa95532_2 defaults
xz 5.2.6 h8cc25b3_0 defaults
zlib 1.2.13 h8cc25b3_0 defaults
zstd 1.5.2 h19a0ad4_0 defaults
Additional Context
No response
After some testing it's one of the following packages causing the error.
colorama=0.4.6=pypi_0 ffmpeg-python=0.2.0=pypi_0 filelock=3.8.0=pypi_0 future=0.18.2=pypi_0 huggingface-hub=0.10.1=pypi_0 idna=3.4=py39haa95532_0 more-itertools=9.0.0=pypi_0 packaging=21.3=pypi_0 pyparsing=3.0.9=pypi_0 pyyaml=6.0=pypi_0 regex=2022.9.13=pypi_0 tokenizers=0.13.1=pypi_0 tqdm=4.64.1=pypi_0 transformers=4.23.1=pypi_0
ffmpeg-python=0.2.0=pypi_0
more-itertools=9.0.0=pypi_0
pyyaml=6.0=pypi_0
regex=2022.9.13=pypi_0
There is a conflict in one of these four.
I can confirm that these PermissionErrors still exist in the latest conda version 23.1.0 running on windows 10 22H2.
In my case it failed to patch the prefix of a .pyd file of wrapt in the site packages folder of the environment I tried to unpack. The issue is also reproducible (with the same file being affected) on dedicated windows CI nodes that do not use any "file monitoring" solutions, which is usually a good candidate for these kinds of problems.
What @vickorian and I have in common is that we try to use conda-unpack.exe. So I tried the documented alternative conda-unpack-script.py with the interpeter that is part of the env I'm trying to unpack. Same problem. Then I used the script but with an independent python interpreter - and it worked reliably.
Hypothesis
The conda-unpack.exe uses the interpreter that is part of the environment to be patched. In other words, the environment is used to patch itself. Maybe that is just a bad idea in general and the robust way is to always use an independent interpreter to perform this job.
conda info
$ conda info
active environment : base
active env location : C:\workflow2\msys2\opt\workflow\workflow_conda_scientific_env_1.6.0.0
shell level : 1
user config file : C:\Users\john\.condarc
populated config files : C:\workflow2\msys2\opt\workflow\workflow_conda_scientific_env_1.6.0.0\.condarc
C:\Users\john\.condarc
conda version : 23.1.0
conda-build version : not installed
python version : 3.9.16.final.0
virtual packages : __archspec=1=x86_64
__cuda=11.7=0
__win=0=0
base environment : C:\workflow2\msys2\opt\workflow\workflow_conda_scientific_env_1.6.0.0 (writable)
conda av data dir : C:\workflow2\msys2\opt\workflow\workflow_conda_scientific_env_1.6.0.0\etc\conda
conda av metadata url : None
channel URLs : https://artifactory.intra.somecorp.com/artifactory/api/conda/conda-anaconda-pkgs-main-virtual/win-64
https://artifactory.intra.somecorp.com/artifactory/api/conda/conda-anaconda-pkgs-main-virtual/noarch
https://artifactory.intra.somecorp.com/artifactory/api/conda/conda-anaconda-pkgs-msys2-virtual/win-64
https://artifactory.intra.somecorp.com/artifactory/api/conda/conda-anaconda-pkgs-msys2-virtual/noarch
https://artifactory.intra.somecorp.com/artifactory/api/conda/conda-anaconda-pkgs-r-virtual/win-64
https://artifactory.intra.somecorp.com/artifactory/api/conda/conda-anaconda-pkgs-r-virtual/noarch
package cache : C:\workflow2\msys2\opt\workflow\workflow_conda_scientific_env_1.6.0.0\pkgs
envs directories : C:\Users\john\AppData\Local\conda\conda\envs
C:\workflow2\msys2\opt\workflow
C:\workflow2\msys2\opt\workflow\workflow_conda_scientific_env_1.6.0.0\envs
C:\Users\john\.conda\envs
platform : win-64
user-agent : conda/23.1.0 requests/2.28.1 CPython/3.9.16 Windows/10 Windows/10.0.19045
administrator : False
netrc file : None
offline mode : False
@axel-kah your solution works.
Use the interpreter of base environment to run the <env name>/Script/conda-unpack-script.py script.
Thanks.
I also second @ZhiZe-ZG's comment.
Specifically, running python <env name>/Scripts/conda-unpack-script.py as described in the directory above the environment name. Running python Scripts/conda-unpack-script.py from within the environment folder still resulted in permission errors.
