Utility method for adding symlinks

[shortercode]

Using addFile resolves symlinks, which isn't always the wanted behaviour. I've created a prototype method for adding an existing symlink to an archive in the same style as addFile.

At the moment error handling leaves a little to be desired, but the method works. It needs more development, but I thought it was time to bring the proposal forward.

[thejoshwolfe]

Thanks for the contribution! I need to do more research into the way existing zipfile readers and writers handle symlinks before i want to commit to supporting a method like this.

To get started, let's talk about how this is supposed to work. Should symlinks to absolute paths be allowed? I suppose so.

Should symlinks values be stored as the file data (as in this PR) or as the variable data in the "UNIX Extra Field (0x000d)" extra field, as specified in the spec? https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT

Things I need to research: If the file mode is set in the External File Attributes to indicate a symlink, then does the strategy of storing the value as the file data result in symlinks "just working" when unzipping? I expect in cygwin it would not work, since I believe cygwin symlinks are stored in some special format involving UTF-16, and this PR encodes the data in UTF-8. I have no idea if this would work in native linux. What happens in native win32?

Zipfile writers to experiment with:

• Linux Info-ZIP (command line zip) with native symlinks
• Cygwin Info-ZIP with cygwin symlinks
• Cygwin Info-ZIP with win32 symlinks
• 7-Zip with win32 symlinks
• Windows Compressed Folder with win32 symlinks
• Mac Archive Utility with native symlinks
• Windows Python zipfile module with win32 symlinks
• Cygwin Python zipfile module with cygwin symlinks
• Linux Python zipfile module with native symlinks
• Java Jar utility - initial research suggests Jar refuses to put symlinks in .jar files.

Zipfile readers to experiment with:

• Linux Info-ZIP
• Cygwin Info-ZIP
• 7-Zip
• Windows Compressed Folder
• Mac Archive Utility
• Windows Python zipfile module
• Cygwin Python zipfile module
• Linux Python zipfile module

[shortercode]

No worries, nice to be able to add something! Both YAZL and YUAZL have been super useful.

I guess it would be useful to explain my use case for this. I have an ubuntu server that takes a project package and returns APKs / Android Studio projects / xcode projects. The responses are being archived with YAZL before being sent down the wire . I had a bug reported last week that some xcode projects weren't build-able, which turned out to be because internal header symlinks had been resolved when I archived them. So the change basically targets this use case, but I hope we can make it more general.

So, basically:

• existing symlinks to relative paths
• only within the archive
• unpacks correctly with mac Archive Utility and unzip

Absolute paths is an interesting case, I suppose there's 2 ways at looking at it. An archived symlink could reference something that exists on the target machine, in which case it's expected. But on the other hand an archive could be intended to work independent of the target environment, in that case it would be better to resolve the symlink. I guess the second case is avoidable provided the developer is aware the link could be absolute.

I wasn't aware that the symlink could be specified in the unix extra field, when I tried looking at the spec last I had tried searching for "symlink" not "symbolic". As I hadn't found anything I thought they left it down to "platform specific" information, to be lost in some unix header file...

I can say that it seems to work on mac as is, I haven't had a chance to check other operating systems yet. I can check windows compatibility tomorrow morning ( GMT ), there's bash on that machine as well so can give that a try. I have to confess I know relatively little about symlinks in general, and I'm probably not set up for running a wide set of compatibility checks. But I'll help in any way I can!

[TooTallNate]

Any progress on this one? I would like to include symlinks in the generated zip files.

[shortercode]

@TooTallNate you can check out my fork, it's fallen behind master but I've been using it for generating zip archives on an ubuntu server. The zip files seem to work for macos and ubuntu at least. I haven't done any work past what is mentioned here, not sure it @thejoshwolfe has done anything for symlinks in the meantime.

[sholsinger]

I have a similar use case to the OP. yazl is used by dwupload and my project references some dependencies w/ symlinks so they can be managed as separate repositories. I need those symlinks to be included as file data. (as if the symlink didn't exist)

[fr-an-k]

I need this for aws lambda (layers with es modules, built with node)

[fr-an-k]

It's actually easy to add a symlink with the existing version; just use:

zip.addBuffer(realPath, symlinkPath, {mode: 0o120777})

This solved my use case; would be good to add this to the documentation. If you want to add existing symlink files you would need to detect and read them yourself anyway. Alternatively, ZipFile could get an option to not follow symlinks when adding files, i.e. add them as symlink files.