markdown.sile icon indicating copy to clipboard operation
markdown.sile copied to clipboard

Remove the luajson manual installation for pandocast

Open Omikhleia opened this issue 2 years ago • 1 comments

The REAME currently states, in the pandocast usage section:

Prerequisites: The LuaJSON module must be installed and available to your SILE environment. This topic is not covered here.

This ought to be a package dependency, without causing an impediment to the user.

From what I recollect, I had issues with non-"sile" scoped dependencies in 0.14.0. To be checked again with 0.14.4 or upper, maybe I was initially wrong too.

Omikhleia avatar Nov 13 '22 14:11 Omikhleia

The actual problem is a bit tougher

  • LuaJSON needs lpeg
  • SILE also, but builds with it
  • But luarocks list on my SILE docker image doesn't show SILE's lpeg being already there, somewhere.
  • Installing LuaJSON via luarocks dependency will thus try to fetch and install it
    • My docker doesn't have build-essentials, so it will fail (no gcc etc.)
      • I could fix it, but anyhow, it means that LuaJSON is then gonna successfully recompile its own version of lpeg
      • I am not even sure which is used at runtime...

Sounds messy, and I am not sure at this point that lots of users will want to use pandocast... So reshuffling the miilestone.

Omikhleia avatar Nov 15 '22 22:11 Omikhleia

  • SILE also, but builds with it

  • But luarocks list on my SILE docker image doesn't show SILE's lpeg being already there, somewhere.

I don't recall seeing this whole issue before until skimming through today's notifications. I see you solved this orthogonally to the core problem and that's fine, but the underlying thing is still an issue. Can you open this as an issue on the SILE repo? I think I saw somebody bump into a variation of this before.

What is happening is that when SILE builds itself a container, we are having it use a few system dependencies, but we're having it vendor all of the LuaRocks stuff it needs. It does this by installing them in /usr/local/share/sile/lua_modules/ instead of the system default path. Then at runtime SILE knows to look there too. It also informs anything running inside SILE about where to find them.

What it doesn't do is make them available to the base system when you just run luarocks .... It could be fixed at least two, probably more ways: we could stop using the vendoring via --use-system-luarocks, then install the necessary Arch packages (or let LuaRocks install the pinned dependencies to the system), or we could add SILE's vendor location to the system Lua manifest, or ...

In any case we can do better and make this more ergonomic.

alerque avatar Sep 27 '24 19:09 alerque

@alerque It's a bit orthogonal indeed, luajson has been fixed for recent versions of lpeg, but without any release to luarocks... So here I went for lunajson as primary/default choice after having tested it....

But yep, you are right, I'm reopening this issue, as a reminder to also report something to SILE regarding lpeg.

Omikhleia avatar Sep 27 '24 19:09 Omikhleia

Reported: https://github.com/sile-typesetter/sile/issues/2118

--> We can close the issue here.

Omikhleia avatar Sep 28 '24 08:09 Omikhleia

That was actually not the issue I had in mind. Installing the Arch Linux sile package in Docker results in a different set of problems than using the SILE Docker images (the former supplies Lua deps via Arch packages, the latter vendors them with LuaRocks).

It's okay though I have a working POC already to fix the latter issue too by making the vendored /usr/local/share/sile/lua_modules visible to the system default luarocks that I will add to this PR so no need for a new issue.

alerque avatar Sep 28 '24 19:09 alerque

Here is the bit I had been thinking about before: sile-typesetter/sile@ca1c9d7 (#2116)

That should bring both methods (our Arch based Docker image with vendored deps or building your own using sile as a package in Arch) up to par such that luarocks list finds everything SILE brought along for the ride.

alerque avatar Sep 28 '24 19:09 alerque