ziglang
Windows does not allow symlinks by default, resulting in some packages unable to be unpacked correctly
Zig Version
zig-windows-x86_64-0.12.0-dev.983+78f2ae7f2
Steps to Reproduce and Observed Behavior
Download my project on windows:
https://github.com/PixelGuys/Cubyz/tree/0c2d309f2af7e66fdb99bb5dac2a2237635151aa
(Sorry for just dumping the entire project here. I don't own windows, so sadly I couldn't create a minimal reproducible. So I only have a report from a user who stumbled over the error)
Run it with zig build run or run.bat(this one automatically downloads the right zig version)
Observe the following error:
error: unable to unpack tarball
.url = "https://github.com/harfbuzz/harfbuzz/archive/refs/tags/8.2.2.tar.gz",
note: unable to create symlink from 'README' to 'README.md': AccessDenied
Expected Behavior
Zig should support packages that contain symlinks. It works fine on Linux by the way.
If it is impossible to create symlinks on windows without permissions, then zig should at least use a workaround, like replacing the symlink file with a copy of the original or something like that.
Weird. Maybe it only happens with certain user permissions or something like that?
Could they try using a newer version of Zig? There were some symlink and tar related PRs merged recently
https://learn.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/create-symbolic-links
Zig does support symlinks on Windows, and the package manager supports them correctly as well. The problem is the OS does not allow them without a setting being toggled.
In this use case, https://github.com/harfbuzz/harfbuzz/archive/refs/tags/8.2.2.tar.gz is a "pristine upstream source" tarball, meaning that the contents are controlled by the harfbuzz project. Those symlinks are present, and they may have semantic meaning, which is why they are represented in the package hash. According to the zig build system, they cannot be replaced with a workaround, because that might mean something different to the package, and it would change the package hash.
Here are two ways for the situation to be resolved:
- The harfbuzz zig package could be provided via the fork strategy instead of the pristine upstream source tarball strategy. In this strategy, one forks harfbuzz, deletes the symlinks and other unnecessary files, adds build.zig to it, and then that is the package that you depend on.
- Zig toolchain could gain more features to control which files from a fetched package are intended to be included in the package hash. This could then exclude the symlinks, which would change the package hash, and make it work fine on Windows. This option depends on #17460 and also depends on a new feature of file filtering being added to build.zig.zon files, which I'm not promising will happen.
https://learn.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/create-symbolic-links
Just to add something on top of that, a quick fix if someone encounters this issue is to enable Developer Mode. I had the AccessDenied issue on a Windows machine and enabling it solved the issue.
Putting aside harfbuzz and the current implementation in the Zig build system... I suspect it would still be a problem even if the hash was calculated before the package was extracted to ensure the package matched what the maintainer expects: if the files are needed then skipping them would break the Windows build, while copying would imply the maintainer should supply a max depth to limit infinite loops.
Symlinks are not enabled on Windows by default because it is a security issue for the platform. Specifically, app developers and even the OS devs often do not have symlinks in mind. Giving users (or yourself) the permission to create symlinks can open up your dev machine to some rare attacks per the warning in the docs.
I am developing and building on a Windows machine and can actively reproduce this issue with libxml2. Specifically when consuming zig-build-libxml2:
.dependencies = .{
.libxml2 = .{
.url = "https://github.com/ianprime0509/zig-build-libxml2/archive/4d1b7db156b0e7a19127c652adefdf27799770ab.tar.gz",
.hash = "122011b13203141cc965cfe6b070ffb5a8835eb906bb2cfd4650dbc17574e6e36fd5",
},
},
Fetch Packages [4/3] libxml2... C:\Users\Jack\AppData\Local\zig\p\122011b13203141cc965cfe6b070ffb5a8835eb906bb2cfd4650dbc17574e6e36fd5\build.zig.zon:16:20: error: unable to unpack tarball
.url = "https://gitlab.gnome.org/GNOME/libxml2/-/archive/v2.11.5/libxml2-v2.11.5.tar.gz",
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: unable to create symlink from 'test/relaxng/ambig_name-class.rng' to 'tutorA.rng': AccessDenied
note: unable to create symlink from 'test/relaxng/ambig_name-class2.rng' to 'ambig_name-class.xml': AccessDenied
Edit: I'm running a recent master branch build, 0.12.0-dev.2208+4debd4338.
Throwing this idea out there and acknowledging it's probably bad / would possibly lead to annoying/bad quirks/errors but:
What if for Windows, Zig could track the symlinks internally itself without creating them and then anytime you use a LazyPath, Zig could redirect the paths that match on known symlink to the actual folder.
https://learn.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/create-symbolic-links
Just to add something on top of that, a quick fix if someone encounters this issue is to enable Developer Mode. I had the
AccessDeniedissue on a Windows machine and enabling it solved the issue.
thank you so much i love u
Corendos commented on Oct 29, 2023
https://learn.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/create-symbolic-links
Just to add something on top of that, a quick fix if someone encounters this issue is to enable Developer Mode. I had the
AccessDeniedissue on a Windows machine and enabling it solved the issue.
The quickfix to enable Developer Mode worked on Windows 10 Version 22H2 (OS Build 19045.5247) (Thanks Corendos).
Interestingly though in documentation: "Default values By default, members of the Administrators group have this right."
I tried using the Windows Terminal in administration mode to call zig build (specifically for ghostty which has build script for the aformentioned harfbuzz and libxml), and that also failed. I don't know if the administrative right is not transient to zig when called from administration terminal though. If that is the case, perhaps a workaround would be to temporarily spawn an elevated instance on retry for specific access denied (per user approval, of course. UAC will probably be tripped as well). If it actually is bugged / documentation out of date, then that won't work of course :^)
This might lead to a pattern of retry-as-admin on Windows for specific AccessDenied errors (such as installing in Program Files), if there are any more in future.
Proposed solution: #22350
I think this is a good solution, but there may still be value in allowing build.zig.zon to specify include/exclude paths for its dependencies.
I just hit this issue, but it wasn't just due to symlinks in the upstream (non Zig) package I'm pulling in--it was due to a mix of both symlinks and characters in paths that aren't allowed on Windows.
If these files were necessary there wouldn't be anything reasonable we could do about that IMO, but it's probably pretty common to only want a subset of the files when using the pristine tarball approach anyway so support for a flag like this doesn't seem too out there.
[EDIT]
Here are two examples of dependencies that I can fetch from Linux and not Windows due to invalid file names:
- https://github.com/systemd/systemd/releases/tag/v257.6
- https://github.com/axboe/liburing/releases/tag/liburing-2.10
