conda
Missing conda-unpack in Conda-Packed Virtual Environment
Checklist
- [x] I added a descriptive title
- [x] I searched open reports and couldn't find a duplicate
What happened?
Issue Overview
I am using conda-pack on Google Colab to compress and extract virtual environments. However, when I compress the environment, the expected conda-unpack file is not generated in the bin folder. The following log output is observed:
- 2025-03-28 19:28:20 [TempCondaEnv.set_env_var] 🔹 LD_LIBRARY_PATH : ['/content/CEnv/app/kohya_env/contents/lib', '/usr/local/lib', '/usr/local/nvidia/lib', '/usr/local/nvidia/lib64']
- 2025-03-28 19:28:20 [TempCondaEnv.check_conda_unpack] ❌ conda-unpack NOT found in the environment.
Steps to Reproduce 1. I create a temporary conda environment using my TempCondaEnv class and install conda-pack via pip inside that environment. 2. Although the LD_LIBRARY_PATH is set correctly, the conda-unpack file is missing from the bin folder.
Code Attachment
Below is the complete code for the TempCondaEnv class that I am using:
`import subprocess import sys from pathlib import Path
def create_temp_venv(env_path, python_version="3.10"): print(f"Creating conda environment at: {env_path}") cmd = ["conda", "create", "-y", "-p", str(env_path), f"python={python_version}"] result = subprocess.run(cmd, capture_output=True, text=True) if result.returncode != 0: print("Error creating conda environment:") print(result.stderr) sys.exit(1) else: print("Conda environment created successfully.")
def install_conda_pack(env_path): print(f"Installing conda-pack in environment: {env_path}") cmd = ["conda", "run", "-p", str(env_path), "pip", "install", "conda-pack"] result = subprocess.run(cmd, capture_output=True, text=True) if result.returncode != 0: print("Error installing conda-pack:") print(result.stderr) sys.exit(1) else: print("conda-pack installed successfully.")
def check_conda_unpack(env_path): conda_unpack_path = Path(env_path) / "bin" / "conda-unpack" if conda_unpack_path.exists(): print("conda-unpack is available at:", conda_unpack_path) else: print("conda-unpack NOT found in the environment.")
if name == "main": temp_env = Path("/tmp/temp_venv") if temp_env.exists(): print(f"Removing existing environment at: {temp_env}") subprocess.run(["rm", "-rf", str(temp_env)]) create_temp_venv(temp_env) install_conda_pack(temp_env) check_conda_unpack(temp_env) `
Question • Based on the code and logs provided, why is conda-unpack not generated as expected when running conda-pack? • Does the use of options like --include conda-unpack or --dest-prefix affect the generation of conda-unpack, or are there other factors at play?
Any insights or suggestions would be greatly appreciated. Thank you!
Additional Context
No response
I encountered a similar issue. It seems that conda-pack silently fails with --format=no-archive option during packing, that is why the conda-unpack script is missing from the packed environment.
(base) peter@DrEvil:~$ conda pack --name=image-testing --output=/home/peter/pack --format=no-archive
Collecting packages...
Packing environment at '/home/peter/miniconda3/envs/image-testing' to '/home/peter/pack'
[########################################] | 100% Completed | 0.6s
(base) peter@DrEvil:~$ conda pack --name=image-testing --output=/home/peter/temporary/image-testing --format=no-archive
Collecting packages...
Packing environment at '/home/peter/miniconda3/envs/image-testing' to '/home/peter/temporary/image-testing'
[############################## ] | 75% Completed | 7min 18.7s
(base) peter@DrEvil:~$ conda pack --name=image-testing --output=/home/peter/temporary/image-testing.tar.gz
Collecting packages...
Packing environment at '/home/peter/miniconda3/envs/image-testing' to '/home/peter/temporary/image-testing.tar.gz'
[########################################] | 100% Completed | 24.2s
(base) peter@DrEvil:~$ conda pack --name=image-testing --output=/home/peter/temporary/image-testing --format=no-archive
Collecting packages...
Packing environment at '/home/peter/miniconda3/envs/image-testing' to '/home/peter/temporary/image-testing'
[############################## ] | 75% Completed | 7min 4.8s
(base) peter@DrEvil:~$ echo ${?}
0
I am packing on WSL. The first command saves the environment locally (inside WSL) successfully, and very quickly. The rest of the commands saves the packed environment to a Windows NTFS drive mounted on /home/peter/temporary. With the --format=no-archive conda pack consistently fails, while with archiving to .tar.gz works. Note that, both creating an archive inside WSL and a .tar.gz to NTFS (where the file is created in a temporary directory in WSL I think) works.
I have successfully used conda-pack many times before, but this image-testing environment that I am trying to pack is new, possibly it has something in it that makes conda-pack fail when archived with --format=no-archive to an NTFS drive that does not support everything that a native EXT4 drive inside WSL does.
I have the same issue inside docker. During building a docker image the packing for the environment therefore the image building fails.
I cannot attach it, so here is the environment configuration:
name: image-testing
channels:
- conda-forge
- defaults
dependencies:
- _libgcc_mutex=0.1=conda_forge
- _openmp_mutex=4.5=2_gnu
- anyio=4.9.0=pyh29332c3_0
- backports=1.0=pyhd8ed1ab_5
- backports.tarfile=1.2.0=pyhd8ed1ab_1
- brotli-python=1.1.0=py39hf88036b_2
- bzip2=1.0.8=h4bc722e_7
- c-ares=1.34.4=hb9d3cd8_0
- ca-certificates=2025.1.31=hbcca054_0
- cachecontrol=0.14.2=pyha770c72_0
- cachecontrol-with-filecache=0.14.2=pyhd8ed1ab_0
- certifi=2025.1.31=pyhd8ed1ab_0
- cffi=1.17.1=py39h15c3d72_0
- charset-normalizer=3.4.1=pyhd8ed1ab_0
- cleo=2.1.0=pyhd8ed1ab_1
- cmake=3.29.4=h91dbaaa_0
- colorama=0.4.6=pyhd8ed1ab_1
- coverage=7.8.0=py39h9399b63_0
- crashtest=0.4.1=pyhd8ed1ab_1
- cryptography=44.0.2=py39h7170ec2_0
- dbus=1.13.6=h5008d03_3
- dicomweb-client=0.59.3=pyhd8ed1ab_1
- distlib=0.3.9=pyhd8ed1ab_1
- dulwich=0.22.8=py39he612d8f_0
- exceptiongroup=1.2.2=pyhd8ed1ab_1
- expat=2.7.0=h5888daf_0
- filelock=3.18.0=pyhd8ed1ab_0
- findpython=0.6.3=pyhff2d567_0
- freetype=2.12.1=h267a509_2
- fsspec=2025.3.2=pyhd8ed1ab_0
- geos=3.13.0=h5888daf_0
- h11=0.14.0=pyhd8ed1ab_1
- h2=4.2.0=pyhd8ed1ab_0
- hpack=4.1.0=pyhd8ed1ab_0
- httpcore=1.0.7=pyh29332c3_1
- httpx=0.28.1=pyhd8ed1ab_0
- hyperframe=6.1.0=pyhd8ed1ab_0
- idna=3.10=pyhd8ed1ab_1
- importlib-metadata=8.6.1=pyha770c72_0
- importlib_resources=6.5.2=pyhd8ed1ab_0
- iniconfig=2.0.0=pyhd8ed1ab_1
- jaraco.classes=3.4.0=pyhd8ed1ab_2
- jaraco.context=6.0.1=pyhd8ed1ab_0
- jaraco.functools=4.1.0=pyhd8ed1ab_0
- jeepney=0.9.0=pyhd8ed1ab_0
- kernel-headers_linux-64=3.10.0=he073ed8_18
- keyring=25.6.0=pyha804496_0
- keyutils=1.6.1=h166bdaf_0
- krb5=1.21.3=h659f571_0
- lcms2=2.15=haa2dc70_1
- ld_impl_linux-64=2.43=h712a8e2_4
- lerc=4.0.0=h27087fc_0
- libblas=3.9.0=31_h59b9bed_openblas
- libcblas=3.9.0=31_he106b2a_openblas
- libcurl=8.8.0=hca28451_1
- libdeflate=1.18=h0b41bf4_0
- libedit=3.1.20250104=pl5321h7949ede_0
- libev=4.33=hd590300_2
- libexpat=2.7.0=h5888daf_0
- libffi=3.4.6=h2dba641_1
- libgcc=14.2.0=h767d61c_2
- libgcc-ng=14.2.0=h69a702a_2
- libgfortran=14.2.0=h69a702a_2
- libgfortran5=14.2.0=hf1ad2bd_2
- libglib=2.80.2=hf974151_0
- libgomp=14.2.0=h767d61c_2
- libiconv=1.18=h4ce23a2_1
- libjpeg-turbo=2.1.5.1=hd590300_1
- liblapack=3.9.0=31_h7ac8fdf_openblas
- liblzma=5.6.4=hb9d3cd8_0
- liblzma-devel=5.6.4=hb9d3cd8_0
- libnghttp2=1.58.0=h47da74e_1
- libopenblas=0.3.29=pthreads_h94d23a6_0
- libpng=1.6.43=h2797004_0
- libsqlite=3.46.0=hde9e2c9_0
- libssh2=1.11.0=h0841786_0
- libstdcxx=14.2.0=h8f9b012_2
- libstdcxx-ng=14.2.0=h4852527_2
- libtiff=4.5.1=h8b53f26_1
- libuv=1.50.0=hb9d3cd8_0
- libwebp-base=1.5.0=h851e524_0
- libxcb=1.15=h0b41bf4_0
- libzlib=1.2.13=h4ab18f5_6
- marshmallow=3.26.1=pyhd8ed1ab_0
- more-itertools=10.6.0=pyhd8ed1ab_0
- msgpack-python=1.1.0=py39h74842e3_0
- ncurses=6.5=h2d0b736_3
- numpy=1.26.4=py39h474f0d3_0
- opencv-python-headless=4.11.0.86=py39hf88036b_1
- openjpeg=2.5.0=hfec8fc6_2
- openssl=3.4.1=h7b32b05_0
- packaging=24.2=pyhd8ed1ab_2
- pbs-installer=2025.3.17=pyhd8ed1ab_0
- pcre2=10.43=hcad00b1_0
- pillow=9.5.0=py39haaeba84_1
- pip=25.0.1=pyh8b19718_0
- pkginfo=1.12.1.2=pyhd8ed1ab_0
- platformdirs=4.3.7=pyh29332c3_0
- pluggy=1.5.0=pyhd8ed1ab_1
- poetry=2.1.2=pyha804496_0
- poetry-core=2.1.2=pyhd8ed1ab_0
- pthread-stubs=0.4=hb9d3cd8_1002
- pycparser=2.22=pyh29332c3_1
- pydicom=2.4.4=pyhd8ed1ab_0
- pyproject_hooks=1.2.0=pyhd8ed1ab_1
- pysocks=1.7.1=pyha55dd90_7
- pytest=8.3.3=pyhd8ed1ab_0
- pytest-cov=6.0.0=pyhd8ed1ab_0
- python=3.9.21=he870216_1
- python-build=1.2.2.post1=pyhff2d567_1
- python-fastjsonschema=2.21.1=pyhd8ed1ab_0
- python-installer=0.7.0=pyhff2d567_1
- python_abi=3.9=2_cp39
- pyyaml=6.0.2=py39h9399b63_2
- rapidfuzz=3.13.0=py39hf88036b_0
- readline=8.2=h8c095d6_2
- requests=2.32.3=pyhd8ed1ab_1
- requests-toolbelt=1.0.0=pyhd8ed1ab_1
- retrying=1.3.4=pyhd8ed1ab_0
- rhash=1.4.5=hb9d3cd8_0
- secretstorage=3.3.3=py39hf3d152e_3
- setuptools=75.8.2=pyhff2d567_0
- shapely=2.0.6=py39hca88cd1_2
- shellingham=1.5.4=pyhd8ed1ab_1
- six=1.17.0=pyhd8ed1ab_0
- sniffio=1.3.1=pyhd8ed1ab_1
- sqlite=3.46.0=h6d4b2fc_0
- sysroot_linux-64=2.17=h0157908_18
- tk=8.6.14=h39e8969_0
- toml=0.10.2=pyhd8ed1ab_1
- tomli=2.2.1=pyhd8ed1ab_1
- tomlkit=0.13.2=pyha770c72_1
- trove-classifiers=2025.3.19.19=pyhd8ed1ab_0
- typing-extensions=4.12.2=py39h06a4308_0
- typing_extensions=4.12.2=py39h06a4308_0
- tzdata=2025b=h78e105d_0
- universal_pathlib=0.2.6=pyhd8ed1ab_0
- urllib3=2.3.0=pyhd8ed1ab_0
- virtualenv=20.30.0=pyhd8ed1ab_0
- wheel=0.45.1=pyhd8ed1ab_1
- wsidicom=0.20.6=pyhd8ed1ab_0
- xorg-libxau=1.0.12=hb9d3cd8_0
- xorg-libxdmcp=1.1.5=hb9d3cd8_0
- xz=5.6.4=hbcc6ac9_0
- xz-gpl-tools=5.6.4=hbcc6ac9_0
- xz-tools=5.6.4=hb9d3cd8_0
- yaml=0.2.5=h7f98852_2
- zipp=3.21.0=pyhd8ed1ab_1
- zlib=1.2.13=h4ab18f5_6
- zstandard=0.23.0=py39h8cd3c5a_1
- zstd=1.5.6=ha6fb4c9_0
prefix: /home/peter/miniconda3/envs/image-testing
