iamgio
Install PDF dependencies without root privileges
I've been trying to setup packaging for Arch Linux (currently here in the AUR), but there is at least one major issue. It seems there is no way to build and package all the dependencies ahead of time. I have the Java stuff worked out to compile to HTML output, but trying to generate a PDF results in something like this:
$ quarkdown c foo.qmd --pdf
<p>foo bar</p><div class="page-break" data-hidden=""></div><h1 id="things">things</h1><ul><li>a list</li><li>of stuff</li></ul>
Exception in thread "main" java.lang.IllegalStateException: Command failed with non-zero exit code:
npm error code EACCES
npm error syscall symlink
npm error path ../lib/node_modules/puppeteer/lib/cjs/puppeteer/node/cli.js
npm error dest /usr/bin/puppeteer
npm error errno -13
npm error Error: EACCES: permission denied, symlink '../lib/node_modules/puppeteer/lib/cjs/puppeteer/node/cli.js' -> '/usr/bin/puppeteer'
npm error at async symlink (node:internal/fs/promises:1004:10)
npm error at async Promise.all (index 0)
npm error at async Promise.all (index 0)
npm error at async #createBinLinks (/usr/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/rebuild.js:396:5)
npm error at async Promise.allSettled (index 0)
npm error at async #linkAllBins (/usr/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/rebuild.js:377:5)
npm error at async #build (/usr/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/rebuild.js:162:7)
npm error at async Arborist.rebuild (/usr/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/rebuild.js:62:5)
npm error at async [reifyPackages] (/usr/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/reify.js:336:11)
npm error at async Arborist.reify (/usr/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/reify.js:141:5) {
npm error errno: -13,
npm error code: 'EACCES',
npm error syscall: 'symlink',
npm error path: '../lib/node_modules/puppeteer/lib/cjs/puppeteer/node/cli.js',
npm error dest: '/usr/bin/puppeteer'
npm error }
npm error
npm error The operation was rejected by your operating system.
npm error It is likely you do not have the permissions to access this file as the current user
npm error
npm error If you believe this might be a permissions issue, please double-check the
npm error permissions of the file and its containing directories, or try running
npm error the command again as root/Administrator.
This is the Java app somewhere trying to fetch and install NPM packages globally at runtime! This is absolutely a non-starter for distributions to package something like this. Somehow it needs to be re-arranged so that either the system can supply the dependencies directly (best) or the dependencies can be fetched/assembled/compiled ahead of time and placed in a vendored directory of some sort where the app can use them having already been installed by the distro package manager.
How do you suggest packaging Puppeteer with the distributable file? @alerque
I would suggest not. Move the npm install stuff behind some sort of command flag (quarkdown install or something like that) and never try to do it automatically when compiling a document. When the user invokes the compile command just assume puppeteer is in the user's $PATH somewhere, that way the system package can just provide it via a dependency that will have it at /usr/bin/puppeteer or wherever is appropriate.
A not-so-nice but workable alternative is provide a way at build time (i.e. next to the gradle tooling) to run the npm install with a specific target argument that generates the node_modules directory you need, and expect that to be available in quarkdown's lib directory or something like that.
Either way quarkdown cannot assume being able to ever write to its own directory location as that will be owned by the system, not the user running it.
Move the npm install stuff behind some sort of command flag (quarkdown install or something like that)
Absolutely reasonable, I will do it so that it covers #54 too. Root permissions will still be required for the installation though.
I believe shipping all the tooling with the distributable file would be the best solution, ux-wise. But with Puppeteer being ~300MB, compared to Quarkdown's ~25MB, this is unlikely to happen.
Puppeteer's size is also the reason why I went for installing the NPM package globally instead of locally. It's going to require root permissions, but at least there can be a single Puppeteer installation for multiple Quarkdown projects.
@alerque could you please test if, after installing Puppeteer by running Quarkdown as root, subsequent PDF compilations run normally without root permission?
Running once as root is a non-starter for distro packages. We must be able to generate all the things that need to be installed as root at package build time. Once the package is built the package manager will take care of placing all the things that need to be places as root, no user apps are allowed to install things into the global system at runtime, not even once.
Just found out (my bad, not an NPM expert) about the --prefix flag that will let Quarkdown install packages globally to a non-restricted location (e.g. <userhome>/.quarkdown or similar). This will fix #54.
@alerque as a last feedback before getting to work: with this enhancement applied, do you still think it is unacceptable to install the package during the first compilation? I'm afraid forcing the user to go for a mandatory install phase would result in a bad ux.
Using the --prefix will help create distro packages as long as we can run it at build time with a special path of the place where the package is being assembled, e.g. $BUILDROOT/usr/lib or whatever it is, and then at runtime the app can find it at /usr/lib/. If the actual prefix gets baked in as the place it needs to be that makes it hard to generate packages.
As to the second part, yes I absolutely do think that apps should never try to fetch or install things at runtime, an first run or otherwise. At the very least prompt for confirmation, but better yet just redirect them to the install command.
BTW I'm still hoping not to include puppeteer in the packaging at all and just depend on it, so a runtime check for puppeteer in the system path should be all that is needed for packaging purposes.
Looking a little bit at the sources (not exhaustively) it looks like you aren't even using the puppeteer CLI at all (which is fine), you just need the puppeteer-core library available to call, no? So I suppose the actual dependencies will end up being nodejs + puppeteer-core, no?
the actual dependencies will end up being
nodejs+puppeteer-core
puppeteer bundles a headless distribution of Chrome, granting consistency across PDF files. puppeteer-core doesn't, as it just exposes the API. It's unlikely to assume all users have (a modern version of) Chrome installed.
It's unlikely to assume all users have (a modern version of) Chrome installed.
In the context of distro packaging, I can guarantee that a modern Chrome is installed. Arch Linux is a rolling release distro, meaning only the latest version of all current packages is a supported state. The same will be true for almost any distro, even not rolling release ones. If the relevant quarkdown package depends on puppeteer and a browser, those dependencies will be current—or at least of the same era as the quarkdown version. For Arch if I make the package depend on a browser it will be the latest build of that at the time of installation.
In any case that variable shouldn't bee too relevant to you, as long as you assume the system can/is providing puppeteer and a matching current headless chrome to go with it.
@alerque in #71 I have deleted any way of installing NPM packages, as you're right: it should be a responsibility of the user/pm to provide the required dependencies. In place of the installation, now there is an explanatory error message.
That said, I'm afraid it's going to be an unpleasant change for the base user. That's why I'm going to postpone merging until I can provide installation via brew/winget to provide the required dependencies out of the box. I hope in a few days.
I'm open to feedback.
#84 should be a good starting point to distribute Quarkdown on package managers, so I can finally merge the PR.
@alerque starting from v1.6.0, Quarkdown is totally package-manager-friendly, as no additional downloads or permissions will be required after the very first installation.