Failed to extract archive on windows

[Krande]

When using pixi and rattler-build v0.18 pixi run dca -c https://repo.prefix.dev/code-aster -c conda-forge on this repository, I get

 ╭─ Running build for recipe: code-aster-17.0.21-py_nompi_release_h_200
 │
 │ ╭─ Fetching source code
 │ │ Downloaded file from https://gitlab.com/krande/src/-/archive/17.0.21.3/src-17.0.21.3.tar.gz
 │ │ Validated SHA256 values of the downloaded file!
 │ │ Extracting tar file to cache: "C:\\Work\\code\\condapackaging\\output\\src_cache\\src-17_0_21_3_9da7ea83.tar.gz"
 │ │
 │ ╰─────────────────── (took 29 seconds)
 │
 ╰─────────────────── (took 29 seconds)
 × error Error building package: Failed to extract archive: failed to unpack `\\?\C:\Work\code\condapackaging\output\src_cache\src-17_0_21_3_9da7ea83\.tmpWso4Cf\src-17.0.21.3\waf`
Error:   × Failed to extract archive: failed to unpack `\\?\C:\Work\code\condapackaging\output\src_cache\src-17_0_21_3_9da7ea83\.tmpWso4Cf\src-17.0.21.3\waf`

Let me know if you need any additional information from me?

Best Regards Kristoffer

[wolfv]

Interesting - does the archive contain symlinks?

[Krande]

yeah, I think there are (at least) a few symlink files at the root of the archive; waf, waf_std, waf_mpi, waf_debug

[wolfv]

Hmm symlinks are tricky on Windows. You can enable developer mode to allow symlinks on Windows without elevated permissions. I don't know how conda-build (or other archive extraction programs) deal with this.

Do you know if this used to work in conda-build?

[wolfv]

https://learn.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development

[Krande]

These files arent used on windows. So they are only relevant for linux. So I guess they're treated as simple binary files?

[wolfv]

Could you try to extract this archive with tar or conda-build on Windows? I am just wondering what they do...

Potentially we could warn (and ignore) symlinks when we cannot create them (instead of failing).

[Krande]

It works without issues when I run conda mambabuild .

Source cache directory is: C:\Work\Miniforge3\envs\mamba-build\conda-bld\src_cache
Found source in cache: code-aster_17.0.21.3_win_9da7ea830c.tar.gz
Extracting download
source tree in: C:\Work\Miniforge3\envs\mamba-build\conda-bld\code-aster_1718963599220\work

And when I do

tar -xvzf /c/Work/Miniforge3/envs/mamba-build/conda-bld/src_cache/code-aster_17.0.21.3_win_9da7ea830c.tar.gz -C /c/Work/

it also works fine. Note that I had to change my directory mapping string when using tar

Update:

Tar does give me some warnings though:

tar: src-17.0.21.3/waf: Cannot create symlink to 'waf_std': No such file or directory
src-17.0.21.3/waf.engine
src-17.0.21.3/waf.main
src-17.0.21.3/waf_debug
tar: src-17.0.21.3/waf_debug: Cannot create symlink to 'waf_variant': No such file or directory
src-17.0.21.3/waf_mpi
tar: src-17.0.21.3/waf_mpi: Cannot create symlink to 'waf_variant': No such file or directory
src-17.0.21.3/waf_std
tar: src-17.0.21.3/waf_std: Cannot create symlink to 'waf_variant': No such file or directory
src-17.0.21.3/waf_variant
src-17.0.21.3/waftools/
src-17.0.21.3/waftools/clangdb.py
src-17.0.21.3/waftools/ext_aster.py
src-17.0.21.3/waftools/mathematics.py
src-17.0.21.3/waftools/med_cfg.py
src-17.0.21.3/waftools/metis.py
src-17.0.21.3/waftools/msvc_lib.py
src-17.0.21.3/waftools/mumps.py
src-17.0.21.3/waftools/official_platforms.py
src-17.0.21.3/waftools/parallel.py
src-17.0.21.3/waftools/parmetis.py
src-17.0.21.3/waftools/petsc.py
src-17.0.21.3/waftools/python_cfg.py
src-17.0.21.3/waftools/runtest.py
src-17.0.21.3/waftools/scotch.py
src-17.0.21.3/waftools/wafutils.py
src-17.0.21.3/wscript
tar: Exiting with failure status due to previous errors

[wolfv]

OK, so it exits with a failure status, but still extracts everything except for the symlinks. I think we can follow that behavior.

[wolfv]

Meaning warn, but still proceed.

[Krande]

Yeah, it seems like it extracts all files. So if that's a possibility then that sounds great!

[Krande]

Hey @wolfv, I think I just ran into another one of these tar extraction with symlink on windows issues.

 │ │ ╰─────────────────── (took 0 seconds)
 │ │ Downloaded file from https://github.com/thelfer/tfel/archive/refs/tags/TFEL-4.2.1.tar.gz
 │ │ Validated SHA256 values of the downloaded file!
 │ │ Extracting tar file to cache: "C:\\Work\\code\\condapackaging\\output\\src_cache\\TFEL-4_2_1_14f27257.tar.gz"
 │ │
 │ ╰─────────────────── (took 0 seconds)
 │
 ╰─────────────────── (took 6 seconds)
 × error Error building package: Failed to extract archive: failed to unpack `\\?\C:\Work\code\condapackaging\output\src_cache\TFEL-4_2_1_14f27257\.tmp97hhn7\tfel-TFEL-4.2.1\README`
Error:   × Failed to extract archive: failed to unpack `\\?\C:\Work\code\condapackaging\output\src_cache\TFEL-4_2_1_14f27257\.tmp97hhn7\tfel-TFEL-4.2.1\README`

When I try to manually extract using tar on windows I it extracts seemingly everything but ends with tar: Error exit delayed from previous errors.

I figured I could take a look, and I did manage to pass the extraction with a slight modification of "extract.rs". See https://github.com/prefix-dev/rattler-build/compare/main...Krande:rattler-build:fix/skip-symlink-extraction.

I'm making a PR also with the suggested fix where we can continue the discussion.