katlogic
Full 5.3 compatibility?
Is it planned?
I understand that fully supporting 64-bit integers in LJX is probably a major undertaking, but I guess a lot of people would like to have integers and bitwise operators in a JIT compiler for Lua.
In short, I have only faux-5.3 in mind - new operators and metamethods. Thats pretty much how Mike's luajit is faux Lua 5.2. It should be "good enough" for most things.
Full compatibility probably never. There is horribly broken branch integer53 where I explored the possibility:
https://github.com/katlogic/ljx/commit/b134f7f00d0c70c44c2e1be9db7635162cea8ede https://github.com/katlogic/ljx/commit/2a33c1765ba0bdf246a637aef08351f1bb51eceb
These are mostly useless (as it was just an experiment and I had no idea what I was doing at the time), however it provides some insight into what needs to be done:
- bend luajit integer narrowing so that it tracks the type correctly, ie no shape-shifting of types in IR
- extend overflow checks in IR to throw error when overflow on integer is detected
This digs into the JIT engine deep enough I'm afraid I won't bother with it, unless there's a volunteer force.
Thank you for the clarifications!
In short, I have only faux-5.3 in mind - new operators and metamethods.
Sounds reasonable... but adding the bitwise operators with Lua-5.2-style numbers would bring subtle compatibility issues, wouldn't it? Or would you assume they apply to LuaJIT-style boxed 64-bit ints only?
subtle compatibility issues, wouldn't it? Or would you assume they apply to LuaJIT-style boxed 64-bit ints only?
Indeed it would be a bit superficial - the MM would fire whenever the operator is applied. Furthermore, if no MM is defined, bit.* equivalent is called - which tries to coerce source operands, and always results FFI int. Basically horrible hackery. I don't have enough mixed Lua 5.3 code at hand to start poking in this, but in the future it will probably show up.
A good rule of thumb for writing future-proof 5.3 code is to not depend on exceptions being thrown during overflows, and not doing explicit type checks in Lua code (on API level, tointeger et al is easy and will work as expected).
The semantic conflicts could be resolved when building LJX as makefile option - it either operates as a frankensteinic monster of "Lua5.1-5.2-5.3" (though a bit fragile, it is super handy for mixing 5.1/5.2 code, the primary reason why I maintain this fork), or as a specific version where conflicting semantics are simply ifdefed out.
I'd like to give heads up that operators are now in "works for me" state, as of https://github.com/katlogic/ljx/commit/37993cfe6d48bc54764736ac34d021881dd5c880
Tester feedback would be welcome. LJX is generally not stable enough for running large software packages, but if you have a simple, but sluggish 5.3 script and you need the JIT kick, this should do the trick for you.
There are few gotchas, though:
- By default, the operators produce 32 bit integers, exact same semantics as bit.* library. This is inentional choice, as such code will not spam GC when in interpreter mode, however might surprise some Lua 5.3 code expecting 64bit integers (fix that code instead!).
- Whenever any bitwise operator encounters a ctype (or *ULL sugared constant), with the exception of bit shift positions, the result is converted to int64. Obviously int64 arguments operate in 64bit mode, just as bit.* does.
- While type semantics are very close to that of of luajit bit.*, the rest follows Lua 5.3 spec - metamethods are called only if arguments fail coercion from string to number (and not being a number, or number-ish ctype).
- Lua 5.3 sillyness (the "hidden" integer type disaster) is not implemented, and I don't plan on doing so. LJX in general is far more permissive compared to 5.3, it will simply coerce doubles to int32 whenever needed, instead of complaining.
Other than that, stuff seems to be working as expected and the code is fully jitted, including metatable dispatch when such table is defined.