[feature] Add capability to download and extract files

[prabirshrestha]

trafficstars

I'm working on photomaps and would like to be able to download files and extract it to a location.

Github repo: https://github.com/protomaps/basemaps-assets/

Here is the file I would like to download: https://github.com/protomaps/basemaps-assets/archive/refs/heads/main.zip

  download_basemaps_assets:
    url: 'https://github.com/protomaps/basemaps-assets/archive/refs/heads/main.zip'
    outputs: 'dest_dir/basemaps-assets.zip'

  extract_basemaps_assets:
    deps:
      - download_basemaps_assets
    zip: 'dest_dir/basemaps-assets.zip'
    outputs: 'public/basemaps-assets'

It is possible to have a native task in moon to download and extract zip and tar.gz files as OS might not have curl/wget or unzip/tar installed by default.

[milesj]

In 1.20, you can now download using extensions: https://moonrepo.dev/docs/guides/extensions#download

However, unpacking isn't a thing. WASM doesn't support file permissions, so this isn't possible right now.

[prabirshrestha]

Hit another use case where I needed to use pdfjs. Their npm package contains the javascript libs but doesn't ship with viewer.html files.

https://www.npmjs.com/package/pdfjs-dist

Have to manually download from https://mozilla.github.io/pdf.js/getting_started/#download which is a zip file.

Might be using something like wasm-flate can be useful here.

[milesj]

The problem is that WASM doesn't support file permissions, so I'm not sure how wasm-flate gets around that. Or maybe it just ignores it entirely.

[prabirshrestha]

You could expose partial WASI. Specifically the WASI FileSystem apis.

[milesj]

We use WASI but Rust only supports the preview1 spec, which doesn't support permissions. We'll have to wait for the next spec to land.