Support for overloaded functions

[marton78]

It would be great to support overloaded functions. An easy way to do it would be to replace the call to rawsetfield after pushing a closure (when registering a method or function) by a call to set_overload. This function is equivalent to rawsetfield if the key is not taken yet. But if the key is taken, it replaces its value by a special function with a table of possible overloads as upvalue:

/*
 * Adds the function at stack top to the table at idx and pops it.
 * If the name is already taken, it adds it as an overload.
 */

void add_overload(lua_State* L, int idx, const char* name)
{    
    if (idx<0)
        idx = lua_absindex(L, idx);

    assert(lua_isfunction(L, -1));

    // try to access the name
    rawgetfield(L, idx, name);

    // if name doesn't exist yet: store it as a regular cfunction
    if (lua_isnil(L, -1))
    {
        lua_pop(L, 1);
        rawsetfield(L, idx, name);
        return;
    }

    // if name exists already, it must be a function
    assert(lua_isfunction(L, -1));

    // stack: -1: old func, -2: new func
    static const char tag[] = "__ovl";

    // test if the current value of idx[name] is already the special function 
    bool needs_setup = true;
    if (lua_getupvalue(L, -1, 2))
    {
        if (lua_type(L, -1) == LUA_TSTRING) {
            lua_pushlstring(L, tag, sizeof(tag)-1);
            needs_setup = !lua_rawequal(L, -1, -2);
            lua_pop(L, 2);
        } else
            lua_pop(L, 1);
    }

    if (needs_setup) {
        // the old closure is a normal function:
        // create new closure of try_overloads with new table
        lua_createtable(L, 4, 0);    // reserve space for 4 overloads
        lua_pushvalue(L, -3);        // push the new function
        lua_pushvalue(L, -3);        // push the original function
        lua_rawseti(L, -3, 1);       // store orig as the table's 1st entry
        lua_rawseti(L, -2, 2);       // store new as the table's 2nd entry
        lua_pushlstring(L, tag, sizeof(tag)-1);
        lua_pushcclosure(L, &try_overloads, 2);
        rawsetfield(L, idx, name);
    } else {
        // the old closure already points to &try_overloads:
        // simply add a new entry to its table
        lua_getupvalue(L, -1, 1);    // get 1st upvalue of the closure
        assert(lua_istable(L, -1));  // it must be a table
        int n = lua_rawlen(L, -1);   // get the number of entries in the table
        lua_pushvalue(L, -3);        // push the new function
        lua_rawseti(L, -2, n+1);     // and store it as the table's next entry
        lua_pop(L, 1);               // pop the table
    }

    lua_pop(L, 2); // pop new and original function
}

The special function for overloads is called try_overloads. Its upvalue is a table of closures, they are all tried until one of them finishes without an error. If all fail, their error messages are concatenated.

/*
 * Callback that is called when a function with overloads is called from Lua
 */

static int try_overloads(lua_State* L)
{
    int nargs = lua_gettop(L);

    // get the list of overloads
    lua_pushvalue(L, lua_upvalueindex(1));
    assert(lua_istable(L, -1));
    const int idx_ovl = nargs + 1;

    // create table to hold error messages
    lua_createtable(L, 8, 0);
    const int idx_err = nargs + 2;
    int nerr = 0;

    // iterate through table, snippet taken from Lua docs
    lua_pushnil(L);  // first key
    while (lua_next(L, idx_ovl) != 0)
    {
        assert(lua_isfunction(L, -1));

        //push arguments
        for (int i=1; i<=nargs; ++i) lua_pushvalue(L, i);

        // call f, this pops the function and its args, pushes result(s)
        int err = lua_pcall(L, nargs, LUA_MULTRET, 0);

        if (err == LUA_OK) {
            // calculate number of return values and return
            int nres = lua_gettop(L) - nargs - 3; // 3: overloads, errors, key
            return nres;
        } else if (err == LUA_ERRRUN) {
            // store error message and try next overload
            lua_rawseti(L, idx_err, ++nerr);
        } else {
            return lua_error(L);  // critical error: rethrow
        }
    }

    lua_Debug debug;
    lua_getstack(L, 0, &debug);
    lua_getinfo(L, "n", &debug);
    lua_pushfstring(L, "All %d overloads of %s returned an error:", nerr, debug.name);

    // Concatenate error messages of each overload
    for (int i=1; i<=nerr; ++i)
    {
        lua_pushfstring(L, "\n%d: ", i);
        lua_rawgeti(L, idx_err, i);
    }
    lua_concat(L, nerr*2+1);

    return lua_error(L); // throw error message just built
}

[marton78]

Note that I check if the 2nd upvalue is equal to the special value "__ovl" to determine if the closure associated to name is already try_overloads. Unfortunately I couldn't come up with an easier way of doing that (lua_topointer for example doesn't work for me)

[vinniefalco]

Marton, this is a very nice solution. I see a problem though - what if the error comes from the function and not the argument conversion? For example:

void foo (int i) {
  if (i%2) lua_error ("i must be an even number");
}

void foo (double d) {
  // function with side effects
}

The Lua code foo (5) would mistakenly call foo (double). This scenario happens almost all the time in my own code, since I register functions which call back into other scripts. Therefore, my registered functions could lead to a lua_error.

I was thinking of adding another property to stack:

template <class T>
struct Stack <double> {
  static inline int getLuaType () { return LUA_TNUMBER; }
};

When an overloaded function is added, we can compute a vector of corresponding Lua types (using the TypeList mechanism). Comparing the argument types in try_overloads() would be easy, just compare the result of lua_type() for each argument against the corresponding element in the vector of each overload. To improve the performance of overload resolution, I would use a small array (8 elements) of function signature type vectors. The array index would correspond to the number of parameters. So if there were two overloads, one taking 0 parameters and the other taking 2 parameters, their type vectors would go into [0] and [2] of this small array. At call time, we lookup the overload set by first indexing on the number of parameters and then looking at the number of overloads. If there is just one overload matching on the number of parameters (commonly the case) then we can go straight to dispatch and not worry about argument types.

We can also detect the situation above where someone registers overloads which are distinct in C++ but ambiguous in Lua. Registering foo (int) and foo (double) should throw an exception - the second overload would never get called in your implementation (or any implementation) since numbers in Lua are all the same type.

[marton78]

I think distinguishing between f(int) and f(double) is a non-issue: it's so easy to call the wrong one even in pure C++ that I don't think anybody would write such code. Also, if one overload fails due to internal reasons, I guess it's fine to try the other overloads as well, since they should fail with a luaL_argerror.

So the only reason to implement a more complicated mechanism is speed. Your idea is not bad, maybe the key in the table of overloads could be some string representation of the Lua-unique function signature: nscTest for number (string, const Test) or what not. However, I'm not sure if that is not breaking a fly on a wheel.

[vinniefalco]

Eventually I want to have a type luabridge::Object which represents any Lua type (similar to what's in luabind). This would be incompatible with using an error in lua_pcall() to determine if the overload resolution was successful. For example:

foo (lua_State* L, Object object, int i)
{
  if (i%2) lua_error (L, "cannot be odd");
}

foo (int i, Object object)
{
   // change state
}

Ignoring the issue of matching the lua_State* (which your case handles, and I think I can handle by just skipping it, consider this lua invocation:

foo (0, 1)

lua_pcall() will fail when foo (lua_State*, Object, int) is invoked but succeed for the invocation of foo (int, Object). This results in side effects. During overload resolution, the candidates from the list of possible functions should be determined and then sorted. Only the best match out of possible overloads would get called. Also, since we have defined an ordering on overloads, any time an overload is registered that would be ambiguous, an exception or error should be generated at the time of registration.

[vinniefalco]

Hey, a string for the function signature vector is a great idea!

[merlinblack]

Just saw you were thinking about a luabridge::Object type thing. I wrote this years ago - http://www.atkinson.gen.nz/talking_with_lua.php

You're welcome to the code, even just to give you some ideas - it worked in concert with the older LuaBridge too!

I liked the luabind::object and wanted something lighter. I haven't touched this in a while however as I went back to luabind, for class inheritance. (I'm keeping an eye on LuaBridge again now though!)

[vinniefalco]

Nigel:

"A lighter luabind::object." Yes, that's exactly what I would like!!!

Thanks for the insights.

Vinnie

On Sun, Nov 11, 2012 at 8:15 PM, Nigel Atkinson [email protected]:

Just saw you were thinking about a luabridge::Object type thing. I wrote this years ago - http://www.atkinson.gen.nz/talking_with_lua.php

You're welcome to the code, even just to give you some ideas - it worked in concert with the older LuaBridge too!

I liked the luabind::object and wanted something lighter. I haven't touched this in a while however as I went back to luabind, for class inheritance. (I'm keeping an eye on LuaBridge again now though!)

— Reply to this email directly or view it on GitHubhttps://github.com/vinniefalco/LuaBridge/issues/9#issuecomment-10277057.

Follow me on Github: https://github.com/vinniefalco

[vinniefalco]

Nigel:

I looked through your code. It looks great! I am definitely going to have to steal it!!!

Thanks

On Sun, Nov 11, 2012 at 9:10 PM, Vinnie Falco [email protected]:

Nigel:

"A lighter luabind::object." Yes, that's exactly what I would like!!!

Thanks for the insights.

Vinnie

On Sun, Nov 11, 2012 at 8:15 PM, Nigel Atkinson [email protected]:

Just saw you were thinking about a luabridge::Object type thing. I wrote this years ago - http://www.atkinson.gen.nz/talking_with_lua.php

You're welcome to the code, even just to give you some ideas - it worked in concert with the older LuaBridge too!

I liked the luabind::object and wanted something lighter. I haven't touched this in a while however as I went back to luabind, for class inheritance. (I'm keeping an eye on LuaBridge again now though!)

— Reply to this email directly or view it on GitHubhttps://github.com/vinniefalco/LuaBridge/issues/9#issuecomment-10277057.

Follow me on Github: https://github.com/vinniefalco

Follow me on Github: https://github.com/vinniefalco

[merlinblack]

Sweet - it's been doing nothing for several years. Glad it's helpful to you. There will be some bits that possibly need tiding up.

[vinniefalco]

Yes I see that the table value proxy could be updated to use the LuaBridge stack, this way it works for userdata generated by LuaBridge.

[vinniefalco]

Nigel's code is being integrated into the "develop" branch now.

[kunitoki]

After ten years, we finally have it. Overloads are with us ! https://github.com/kunitoki/LuaBridge3/pull/38