cannot build in sandboxed env because npm dependencies aren't accessible

[floppym]

What is the issue?

As a distro packager, it would be easier to enable the web UI if the all of the necessary files were included in the source tarball.

On Gentoo, we build packages inside a sandboxed environment with no access to the internet by default. It is problematic to fetch files in the middle of the build process.

Which application of Transmission?

Web client

Which version of Transmission?

4.0.0-beta1

[ckerr]

I hear what you're saying but I can't believe that bundling the dependencies in a tarball is the solution. That would make the tarball many times larger than it currently is, for a marginal benefit, since those dependencies are only needed when building. On top of that, npm deps are so fast-moving that whatever we bundle will quickly by out-of-date.

DYK if there's any prior art from other monorepos that ship with something using npm deps?

[mikedld]

Don't know how good this would be, specifically in terms of Gentoo that wants to build everything from source, but could avoiding the build be a viable solution too? E.g. we could provide a separate tarball as a release artifact, containing prebuilt web files, which people would use if they don't want to build themselves and they trust us with it (will optionally mean us somehow signing it).

[floppym]

I would be ok with a file containing pre-built web files. I assume the same files could be used regardless of the CPU arch. I think the major downside is we would lose the ability to easily apply patches for back-ported bug fixes or user customization.

[floppym]

I hear what you're saying but I can't believe that bundling the dependencies in a tarball is the solution.

If you don't provide some solution upstream, I can probably hack something together downstream by pre-fetching the deps and building my own tarball. I can't rely on the npm repos always being up/available, and our packaging system (Portage) really doesn't support that anyway.

That would make the tarball many times larger than it currently is, for a marginal benefit, since those dependencies are only needed when building.

Isn't building the software is the primary purpose of a source tarball? If I just wanted to look at the code, I could browse it on github. I guess you mean it's only useful when building the web client, which you don't expect many people to do?

On top of that, npm deps are so fast-moving that whatever we bundle will quickly by out-of-date.

~~I think yarn.lock ensures that the deps don't magically update anyway. I think you would need to run yarn up to fetch newer deps, and CMakeLists.txt doesn't currently do that.~~

Edit: I don't actually have any experience using yarn, so disregard my comment above.

DYK if there's any prior art from other monorepos that ship with something using npm deps?

Chromium (the web browser) includes the necessary node modules in the source tarball. However, they have a complex homegrown dependency system, and the source tarball for chromium-108 is 1.6 GB (even compressed), so that might not be the best example to follow.

I don't really have any better examples.

[mikedld]

This may be something worth looking into: https://classic.yarnpkg.com/blog/2016/11/24/offline-mirror/. The way it could work is that we distribute a separate tarball with web deps as part of the release which could then be unpacked and used as an offline mirror. If we don't want to do this for some reason, maybe Gentoo (and other distros?) would consider creating such tarball themselves (but I guess if it's more than one distro then we're all better off doing it once ourselves for everyone to use).