lit icon indicating copy to clipboard operation
lit copied to clipboard

How to build binary modules?

Open devurandom opened this issue 10 years ago • 6 comments

I would like to use lit as a dependency management tool for non-luvit Lua modules. Since some of them are binary modules (i.e. *.so files), I need to build them first. Thus I would like to call into Lake to produce these files, and then give control back to lit to install them in the appropriate locations. Is this possible with lit? How?

devurandom avatar Aug 28 '15 14:08 devurandom

You can't run any scripts as part of publish or install, but there is some logic to use pre-built binaries. It's documented at https://github.com/luvit/lit/wiki/Publishing-Compiled-Code

I hope this works for your use case, let me know.

creationix avatar Sep 02 '15 17:09 creationix

Let me see whether I understood correctly how lit works:

  1. I run make to build the binaries.
  2. I run lit publish . to build a zipfile of everything listed in package.lua.
  3. When some Lua code requires that package, init.lua will be called.

I am trying to achieve something like Python's "pip" (i.e. build the binaries when running pip install on the enduser's system). So to package my code in a way that does not require me to ship binary files for all kinds of operating systems, I would skip step 1 and publish a package that contains an init.lua that downloads the code (or alternatively: make the published package contain the code already), compiles it, requires the binary module and hands the resulting table back as it's own return value?

devurandom avatar Sep 02 '15 19:09 devurandom

Yes, this would be a neat feature, but the official luvi doesn't even distribute the header files required for this in the distribution.

I do publish a full source tarball for each luvi release so I suppose lit could download this and cache it locally for the current luvi version. But then we also need some standard way to build addons, probably cmake based to match the luvi build system.

It would be great if the precompiled binaries could work in addition to this new system so compiling from source wouldn't be required for all platforms. It's much harder to setup a proper build toolchain on windows than it is in linux for example, but on the other hand, windows has much better ABI compatibility between versions so pre-built binaries work well there.

creationix avatar Sep 03 '15 14:09 creationix

I'd be willing to add this to lit, we just need to discuss and design how it should work. Python's PIP was mentioned, another relevant prior art is node-gyp and how it interacts with node and npm.

creationix avatar Sep 03 '15 14:09 creationix

Meanwhile I rediscovered luarocks, which supports this: build={type=builtin|make|cmake}. But I also read that you don't want to use it, at least for packaging. Then there is also slingshot, but that is basically a wrapper around luarocks and autotools.

TBH, I just remembered lit, when I was searching for a way to build and distribute some "legacy" Lua (binary) module, not a Luvit module. So I may be wrong here anyway... Is there a document comparing lit and luarocks (potentially combined with slingshot)? What to expect from lit? What should be used for which purpose, etc?

What I was trying to achieve was to package lua-linenoise, so I could easily use it in my own program - i.e. an end-result like:

foo-package-manager install --local linenoise
echo 'local L = require "linenoise"' > my-script.lua
lua my-script.lua # works!

devurandom avatar Sep 03 '15 15:09 devurandom

Lit will work as well as you don't mind pre-publishing all binaries for platforms you support as well as being having the module installed to $CWD/deps/$NAME.

creationix avatar Nov 03 '15 20:11 creationix