Using an LPEG grammar, "error converting lua table to userdata"

[alerque]

I have an existing app that used to run in Lua and now inside a Rust wrapper app using mlua.

I have an LPEG grammar on the Lua side I'd like to use and :match() on directly from Rust. I'm having trouble with return types though and it isn't giving me a LuaTable like I would expect. All this code is targeting mlua v0.9.x.

Here are some bits and bobs involved, first the grammar:

bits.cliuse = Ct(Cg(module, "module") * Cg(useparams^-1, "options"))

Here is what that grammar would look like in use:

$ sile -e 'a="foo[bar=baz]"; b = SILE.parserBits.cliuse:match(a); SU.dump{ a, type(a), b, type(b) };os.exit()'
{
  [=[foo[bar=baz]]=],
  "string",
  {
    module = "foo",
    options = {
      bar = "baz"
    }
  },
  "table"
}

So given a string, the match method returns a table.

What I want to do with it is iterate over some strings and stuff the resulting matches in a table. I got this Rust code to work:

for module in modules.iter() {
    let module = lua.create_string(module)?;
    lua.load(chunk! {
        local spec = SILE.parserBits.cliuse:match($module);
        table.insert(SILE.input.uses, spec)
    })
    .eval::<()>()?;
}

As expected I can iterate ofer the list of input modules and get the expected table as a result:

$ sile -u 'foo[bob=trink]' -u 'apple[tree=6,foo=bar]' -e 'SU.dump(SILE.input.uses);os.exit()'`
{
  {
    module = "foo",
    options = {
      bob = "trink"
    }
  },
  {
    module = "apple",
    options = {
      foo = "bar",
      tree = "6"
    }
  }
}

But this is just dropping into a Lua environment and letting it parse and run some code. I already know that worked on the Lua side. What I expected to work was something that kept all the moving parts in Rust:

let parser_bits: LuaTable = sile.get("parserBits")?;
let cliuse: LuaAnyUserData = parser_bits.get("cliuse")?;
let input_uses: LuaTable = sile_input.get("uses")?;
for module in modules.iter() {
    let module = lua.create_string(module)?;
    let spec = cliuse.call_method::<_, LuaTable>("match", module)?;
    let _ = input_uses.push(spec);
}

This compiles, but given the same inputs as before it bubbles up an error at runtime:

$ sile -u 'foo[bob=trink]' -u 'apple[tree=6,foo=bar]' -e 'SU.dump(SILE.input.uses);os.exit()'
Error: userdata is not expected type

Having debugged it a little bit, the error is not happening after spec is created when it tries to stuff it in the table, it's actually the call_method() that is raising the error.

What should the proper use of .call_method() be on an LPEG grammar so as to use the :match() method and get something useful in return?

[khvzak]

Do you have any code that I can run? On the first sight everything looks ok.

PS you can attach extra context to Lua errors to make them more friendly, eg:

cliuse.call_method::<_, LuaTable>("match", module).context("failed to call `cliuse:match()`")?;

[alerque]

Sure, you can run the real thing. If that's too much trouble I can try to work up an MWE another day, I definitely won't get around to it tonight.

The project is here. The master branch has the currently working Lua chunk! {} approach. The more-rusty branch on my fork has the change discussed here.

If you use nix at all (the tool not the OS) the easiest way you can run each variant is with the Flake. I think you have to enable Flake support in your user config, but once allowed to run any flakes you can run any branch/commit/tag of the project just from the GH URL:

$ nix run github:sile-typesetter/sile/master -- -q -u 'foo[bar=baz]' -e 'SU.dump(SILE.input.uses);os.exit()'
{
  {
    module = "foo",
    options = {
      bar = "baz"
    }
  }
}%

$ nix run github:alerque/sile/more-rusty -- -q -u 'foo[bar=baz]' -e 'SU.dump(SILE.input.uses);os.exit()'
Error: userdata is not expected type

Also if you clone locally you can cd into the project and run nix develop to get a development shell with all the dependencies worked out.

Whether you take the Nix route to satisfy dependencies or do that on your own the first time you use the checkout you can run ./bootstrap.sh, thereafter checking out the branch or making other changes can be followed with ./configure --enable-developer-mode --without-manual && make to get a working binary you can test in the project with ./sile .... After configuring cargo run --features luajit -- ... should also work for debug runs.

I can try to replicate something with a minimal cargo project and some LPEG code, but it will take a little while to setup.

[khvzak]

I debugged the issue a bit and the problem is that cliuse is a foreign userdata so mlua v0.9 cannot retrieve metadata for it (for safety reasons). It's actually fixed in v0.10. I probably need to backport some changes.

[alerque]

I've confirmed the way I expected this to work actually does in v0.10.0, so we're good here. Again congrats on the release, it's a nice one.