love
love copied to clipboard
love2d
Package LuaJIT debug extensions with LÖVE
Hi,
Is it planned to properly integrate LuaJIT 2.1 by including all its Lua modules (jit.*) on all supported platforms ? I think this has been an overlook of previous LÖVE versions and with the switch to LuaJIT 2.1 it is an opportunity to fix it.
The jit.* modules are necessary, especially for performance analysis (e.g. high-level API for the built-in profiler). They are tied to a specific build of LuaJIT (by version and built vmdef.lua). Thus they are not third-party resources that can be distributed separately and it is even more problematic because LuaJIT seems to move away from explicit versioning to commit hashes (but it only aggravates the original issue).
If it's too large size-wise, another option could be a debug-build binary release where they are included?
(i can attest to the difficulty of getting them separately if one doesn't build löve themselves from source... and even then, it is possible that it would need to be enabled for those modules to be built as well, although i'm not sure on this part.)
We wouldn't be able to include them in love's source like we do with every other Lua module LÖVE comes with, because we don't necessarily use the same LuaJIT commit on every platform (nor could we guarantee it).
We'd have to come up with a way to distribute them alongside the executable in a manner that lets love.filesystem load them. It would need to be done separately for each platform LÖVE supports. Keeping them outside the executable would also let people swap them out if they choose to replace the LuaJIT dll with a different one (which people can do without recompiling love).
That being said, as far as I understand they are typically meant for debugging LuaJIT's internals rather than being part of LuaJIT's core extensions, which is why they aren't inside LuaJIT itself and are only installed with the standalone executable. They have some use cases with LÖVE of course, but they're intentionally an extra thing on top of all of LuaJIT's built-in modules, rather than a part of it. It's not something we overlooked, it was a choice by LuaJIT to prevent easy use of those modules when LuaJIT is embedded with an app as is the case with love.
If you feel strongly that the modules should be more easily usable in embedded situations, it's probably worth raising that on the LuaJIT issue tracker too.
[...] it was a choice by LuaJIT to prevent easy use of those modules when LuaJIT is embedded with an app as is the case with love
I don't see how this has anything to do with LuaJIT's design. With LuaJIT and Lua, as the host program, you can load (or not) what you want; the fact that they are pure Lua modules doesn't make them less usable (quite the contrary to me). (A more important question may be: what makes it so hard to include pure Lua modules with LÖVE ?)
If it's not an overlook, then it's a deliberate choice of not including them and it means this issue is only about a conflicting POV. To me, they are mandatory to exploit LuaJIT's full potential.
E.g
they are typically meant for debugging LuaJIT's internals
They have some use cases with LÖVE of course
is definitely not how I would resume the usefulness of the jit.* modules.
I can, of course, build the jit directory for a specific commit of LuaJIT and a specific platform (I don't know if vmdef.lua generates platform-dependent data), but it's a burden.
And that burden will probably prevent most users from using the built-in profiler.
If it's too large size-wise, another option could be a debug-build binary release where they are included? -- zorggn
Probably not much better than not having it at all, and then we would be weirdly thinking about profiling a debug build. If it is not transparently available in LÖVE, it would probably be a better solution to just distribute the built jit directory with each LÖVE release for each platform to let everyone include it in their app.
I don't understand what makes it hard to include the modules for each platform, but I (can) only consider the Windows (everything in the final app, next to the *.dll) way and the POSIX/distributions way (a specific directory where LÖVE stores resources).
I don't see how this has anything to do with LuaJIT's design. With LuaJIT and Lua, as the host program, you can load (or not) what you want; the fact that they are pure Lua modules doesn't make them less usable (quite the contrary to me).
Every documented LuaJIT module and every documented Lua module is available to require when LuaJIT is loaded.
These extra internal debug libraries are not available in that situation and not documented (the only official online documentation related to them is for the standalone executable's command line arguments, which love doesn't use). They need extra free-floating Lua files that are tied to the internal commit-specific details of the LuaJIT library despite being free-floating.
If LuaJIT wanted these modules easily accessible by every user of the library, they'd be part of the actual LuaJIT library itself (similar to the main jit, ffi, and bit modules) rather than free-floating files separate from the library. As-is they're distributed with the LuaJIT standalone executable rather than being distributed as part of the LuaJIT library.
I don't understand what makes it hard to include the modules for each platform
If you have a proposal or pull request for love's build setups that automatically bundles these files with the executables in a manner that meets the following requirements, I'm interested. As you can see it's not as trivial as adding other Lua modules:
- Each platform needs to bundle the Lua files in a manner that fits with the folder hierarchy of apps in the platform. For example in Windows they can be next to the executable, in macOS they need to be somewhere in the .app (perhaps in Resources), etc.
- They need to be automatically moved into those locations when building, it can't be a manual step.
- They should be copied from the precise LuaJIT version that love links with on that platform. This means they need to be available along with prebuilt LuaJIT binaries in situations where we use those, so any build scripts for making those need to be updated accordingly.
- They should be removable/replaceable by users of love without rebuilding love, because LuaJIT can be replaced and because many developers won't want to ship them with their games.
- love.filesystem needs to be able to load them on every OS at runtime.
My point earlier is that no other part of any of love's dependencies or code needs this, but these debug modules do for the reasons outlined earlier. Therefore it's not a change that can use existing functionality in love's build setups and needs special handling compared to any other module that LuaJIT and Lua provides itself.
Considering it would be easy to forget to remove them when distributing a game but in my opinion most games should not ship with them, I'd rather either have separate downloads for them, or update every love fuser/builder to strip them (perhaps with an opt-in to keep them).
Also, a side note: many of these internal debug modules use io.write (which needs platform-dependent file paths to use) - and the high level profiler jit.p only seems to have a single line of code that relies on vmdef. It might not be a lot of work for someone to fork some of those modules to use less platform and jit-internal dependent code, which would avoid needing love to bundle them for you.
I think this is a good example of something simple not going to be simple because of all the complexifying choices and constraints made by multiple parties along the lines. It's possible to do it properly and I value generalization, i.e. to give to all LÖVE users the abstraction over this, but it doesn't seem to fit with the framework/community intents and thus is not really worth the generalization effort.
Distributing the built files with each platform release and letting users embed them as they see fit seems to be the most effective solution.
I may be the only one who voiced concerns about this issue, thus I want to add that as long as the LuaJIT's version is known (which should not require anything from LÖVE ultimately, see https://github.com/LuaJIT/LuaJIT/issues/711), it is not as important as I first though.
Having those modules in end-user environments would be nice, but it's not essential.
I'm not quite sure about iOS or macOS, but I think this is sort-of shipped already.
- For Linux AppImage, this is built-in. You don't have to do anything.
- For Windows, the JIT modules are available in our Discord server (search for "jit modules has:file") and in our releases page.
- For Android, the prebuilt contains JIT modules in each of the ABI subdirectory. https://github.com/love2d/love-android/tree/11.4/love/src/jni/LuaJIT-2.1/android
For Windows, the JIT modules are available in our Discord server (only at the moment).
They're here: https://github.com/love2d/love/releases/tag/11.4
As well as in the artifacts of CI builds.
Hi,
Just to update that, from my end, the issue is fixed as it can be. But maybe there is a reason it is still open.
Also, thank you for taking time about the issue. I tend to react negatively with conflicting point of views, thus I hope I didn't contribute too much (with negativity) to the burden of maintaining a FOSS project.