Documentation: C API

[zeux]

trafficstars

We currently don't have documentation for C API and simply call out to https://www.lua.org/manual/5.1/manual.html#3 in README. It would make sense to have a separate page, that initially could be similar to our syntax page - link to the 5.1 manual and note the differences separately.

[zeux]

List of differences off the top of my head:

• Loading bytecode into VM is done via luau_load and bytecode needs to be compiled from source via luau_compile (possibly offline)
• lua_pushcfunction and lua_pushcclosure accept an extra argument, debugname, which is recommended to set if the name of the function should appear in stack traces
• lauxlib.h header is replaced by lualib.h header
• luaL_ref/luaL_unref are replaced with lua_ref/lua_unref; lua_ref takes the stack index of the value
• __gc is not supported; code that uses __gc for userdata objects should use lua_newuserdatadtor
• lua_getstack doesn't exist; instead, lua_getinfo accepts the stack frame index as an argument
• Userdata objects don't support fenv (5.1) or uservalues (5.2+)

[zeux]

List of extra features off the top of my head:

• Luau expects sandboxed global tables which prevents monkey-patching and enables a host of optimizations in the VM; the easiest way to do this is to call luaL_sandbox on the global state after initialization and global setup, and call luaL_sandboxthread on each top-level script thread that is created to run script code. Luau will work without it but will be slower.
• Instead of using luaL_checkudata to work with safely typed userdata, Luau optionally provides support for tagged userdata that can be used by assigning a unique short tag to each type (limited to 64 by default) and validated via lua_touserdatatagged. When working with tagged userdata, __gc is replaced by lua_setuserdatadtor
• When exposing objects from the host, to accelerate method calls __namecall metamethod may be defined; it is invoked directly when obj:Method(args) is called, with method name available via lua_namecallatom
• When working with method/field names from the host, it's possible to embed a short (16-bit) unique id into strings that match the exposed API surface to avoid comparing strings in the host implementation; this requires overriding useratom callback via lua_callbacks to compute the atom, and lua_tostringatom / lua_namecallatom to retrieve it.

[JDaance]

No lua_setfenv for userdata is a difference, not many people should care but it bit me when using common lua libs like lpeg.

lua_getstack is removed, and the arguments for lua_getinfo has changed to take the stack level instead of the debug struct from lua_getstack. It was a good change in my opinion, all code I changed got cleaner.

A bunch of #defines are missing compared to standard lua. Here is some of the stuff I added in my lauxlib.h (which is also missing)

#define LUA_VERSION_NUM	501
#define LUA_NUMBER_DOUBLE
#define luaL_reg	luaL_Reg

#define luaL_putchar(B,c)	luaL_addchar(B,c)
#define luaL_checkint(L,n)	((int)luaL_checkinteger(L, (n)))
#define luaL_optint(L,n,d)	((int)luaL_optinteger(L, (n), (d)))
#define luaL_checklong(L,n)	((long)luaL_checkinteger(L, (n)))
#define luaL_optlong(L,n,d)	((long)luaL_optinteger(L, (n), (d)))
#define luaL_openlib(L, libname, l, nup) luaL_register(L, libname, l)

LUALIB_API void (luaL_openlib) (lua_State *L, const char *libname,
                                const luaL_Reg *l, int nup);
LUALIB_API int (luaL_typerror) (lua_State *L, int narg, const char *tname);
LUALIB_API int (luaL_ref) (lua_State *L, int t);
LUALIB_API void (luaL_unref) (lua_State *L, int t, int ref);
LUALIB_API const char *(luaL_gsub) (lua_State *L, const char *s, const char *p,
                                                  const char *r);
LUA_API lua_Alloc lua_getallocf (lua_State *L, void **ud);

Not saying you should add any of this stuff, but these are the C api changes I ran into

Edit: Removed luaL_checkstring/luaL_optstring

[zeux]

luaL_checkstring/luaL_optstring do exist in lualib.h. Good point about lua_getstack and other missing macros / functions.

[petrihakkinen]

Another new function that could warrant documentation is luaL_findtable.

btw. should lua_isvector, luaL_checkvector and luaL_optvector be added? I find these handy.

[zeux]

Yeah we should add functions for vectors.

[petrihakkinen]

Ok, I'll add them!

[zeux]

luaL_findtable actually comes from 5.1 but it appears to not be documented in the 5.1 manual and it seems to be removed in future versions. It's sort of an internal implementation detail of luaL_register but it supports dotted paths which is pretty unusual... I feel like this is a function that might not survive after we actually define the new string-based require semantics officially instead of as a stop-gap implementation in REPL but we'll see.

[zeux]

The difference with lua_newuserdata wrt vanilla isn't super intentional as tags are meant to be optional; we'll probably fix that by making newuserdata a macro. There's still the difference with later versions (no uservalues support) and 5.1 (no userdata env support) but I don't think that's going to change so would be good to document - I'll add this to my earlier comment.

[petrihakkinen]

I agree luaL_findtable is unusual. Perhaps it should be refactored into internal function in REPL?

[JDaance]

I noticed now that the signature for lua_Alloc is different from 5.1, adding lua state as first argument

[zeux]

Ah, so it is. I don't think we actually are using this, we should change back to match upstream better.

[JDaance]

Ah, so it is. I don't think we actually are using this, we should change back to match upstream better.

Yes that would be great 👍 looking forward to removing my workaround 🙂

[aloofbynature]

I'm not really sure if this is the right spot to ask this question but I'm going for it. Is it possible to create C modules that can be accessed in luau via luaL_register?

I've been trying to follow the Lua C api docs from the 5.1 reference manual but luau deviates a bit. I see there's a lua_pushcfunction method exposed that I could probably use to set a global function but the luaL_register just looked like a neater implementation.

I'm not sure if this matters but I am compiling on a mac using cmake. So I am generating a .dylib instead of an .so like the docs mention, will that be a problem? Right now I have a .dylib generating in the same directory as my luau executable but when I run the REPL and require('module') I get an error along the lines of stdin:1: invalid argument #1 to 'require' (error loading gfx) stack backtrace: [C] function require stdin:1

[zeux]

You can use luaL_register just fine. The default require implementation in REPL doesn't look for globals, it always assumes the input is a file path. However, once you call luaL_register you can simply access the global, so instead of local gfx = require("gfx") you can just use gfx.foobar if this was part of the table you passed to luaL_register(L, "gfx", lib).

[kkukshtel]

Just to add some context here for people who find this repo and are looking for a way to setup Luau from C bindings, you can check out the Program.cs file in Luau.NET for basic setup. It's in C# but the function binding names are all the same as the C counterparts. The main setup afaik is:

var script = "print('hello world from luau')"; //some script text
var scriptBytes = System.Text.Encoding.UTF8.GetBytes(script); //c# to get script bytes
Luau.lua_CompileOptions compOpts = default; //setup the compile options
nuint outsize = 0;
var L = Luau.Luau.luaL_newstate(); //create a new state
Luau.Luau.luaL_openlibs(L); //open the lua libs
var chunkname = System.Text.Encoding.UTF8.GetBytes("test");
fixed (byte* ptr = scriptBytes, chunk = chunkname)
{
    //compile the script
    var compiledBytecode = Luau.Luau.luau_compile(
        (sbyte*)ptr,
        (nuint)(scriptBytes.Length * sizeof(byte)), 
        &compOpts,
        &outsize
        );
    //load the script into luau
    int result = Luau.Luau.luau_load(
        L, 
        (sbyte*)chunk, 
        compiledBytecode, 
        outsize,
        0);
    Console.WriteLine(result);
}
var status = 0;
bool run = true;
while (run)
{
    //keep running lua while we aren't erroring
    status = Luau.Luau.lua_resume(L, null, 0);
    switch ((Luau.lua_Status)status)
    {
        case lua_Status.LUA_ERRRUN:
            var s = Luau.Luau.macros_lua_tostring(L, -1);
            Console.WriteLine(Marshal.PtrToStringAnsi((IntPtr)s));
            var trace = Luau.Luau.lua_debugtrace(L);
            Console.WriteLine(Marshal.PtrToStringAnsi((IntPtr)trace));
            Luau.Luau.lua_close(L);
            run = false;
            break;
    }
}

[aaaddd4747]

Just to add some context here for people who find this repo and are looking for a way to setup Luau from C bindings, you can check out the Program.cs file in Luau.NET for basic setup. It's in C# but the function binding names are all the same as the C counterparts. The main setup afaik is: {code snippet removed}

great info as im trying to do exactly that! unfortunately luaL_newstate() doesnt seem to actually exist anywhere as far as i can tell. for now, in order to get anything running at all i had to take a function from the conformance test in order to get lua_newstate to work as it takes args not described anywhere as far as i can tell.

[kkukshtel]

Just to add some context here for people who find this repo and are looking for a way to setup Luau from C bindings, you can check out the Program.cs file in Luau.NET for basic setup. It's in C# but the function binding names are all the same as the C counterparts. The main setup afaik is: {code snippet removed}

great info as im trying to do exactly that! unfortunately luaL_newstate() doesnt seem to actually exist anywhere as far as i can tell. for now, in order to get anything running at all i had to take a function from the conformance test in order to get lua_newstate to work as it takes args not described anywhere as far as i can tell.

It's in the VM libs here. I think this is the function you want to call, the primary lua.newstate function takes in a lot of params, so I think it's what you want if you're creating a "normal" Lua state. Because Luau sits on top, it does state management for you, hence the parameterless luaL.newState function. @zeux can obviously check my understanding here if that's the case. I was primiarly using the tests in this repo to understand how things were meant to be invoked so Luau.NET just mimicks that pattern I saw.

[zeux]

Yeah, luaL_newstate is a good default to use. It's available via lualib.h just as all other functions with luaL prefix (Lua 5.x uses lauxlib.h but we tried to make sure all public headers start with lua for consistency, hence the naming change).

[Nolram12345]

Is there any like "hello world" example that shows how to properly use Luau in an embedded context? It seems a lot less of a drop-in Lua replacement than anticipated...

[kkukshtel]

Is there any like "hello world" example that shows how to properly use Luau in an embedded context? It seems a lot less of a drop-in Lua replacement than anticipated...

See my comments above. They are in C# but easily convertible to whatever, the function names are all that matters.

[Nolram12345]

Thank you, but a sample also outlining just how to properly set up a build for embedding, what headers should be included etc. in C/C++ would also be immensely helpful, as there is basically no documentation at all on how to do this with Luau.

[forenoonwatch]

Is there any documentation on the thread safety of various parts of the C API, and how they interact in a concurrent context?