Python exceptions can't be handled from Lua scripts using pcall

[kmike]

>>> import lupa; lua = lupa.LuaRuntime()
>>> pcall = lua.eval("pcall")
>>> def py_func(): raise Exception("python exception")
... 
>>> lua_func = lua.eval("function () error('lua error') end")
>>> pcall(lua_func)
(False, u'[string "<python>"]:1: lua error')
>>> pcall(py_func)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "lupa/_lupa.pyx", line 507, in lupa._lupa._LuaObject.__call__ (lupa/_lupa.c:7743)
  File "lupa/_lupa.pyx", line 1245, in lupa._lupa.call_lua (lupa/_lupa.c:17287)
  File "lupa/_lupa.pyx", line 1252, in lupa._lupa.execute_lua_call (lupa/_lupa.c:17381)
  File "lupa/_lupa.pyx", line 231, in lupa._lupa.LuaRuntime.reraise_on_exception (lupa/_lupa.c:4144)
  File "lupa/_lupa.pyx", line 1386, in lupa._lupa.py_call_with_gil (lupa/_lupa.c:18673)
  File "lupa/_lupa.pyx", line 1352, in lupa._lupa.call_python (lupa/_lupa.c:18407)
  File "<stdin>", line 1, in py_func
Exception: python exception

[kmike]

Is it a bug or an intended behaviour?

[scoder]

At least, it doesn't crash. These things used to. :)

Note that the pcall(lua_func) essentially just unpacks both Lua functions and calls one with the other as argument inside of Lua, so the Lua error doesn't actually pass through Python at all here.

The other way is tricky because supporting pcall(), correct me if I'm wrong, would mean that we translate a Python exception into a Lua longjmp() error to pass it on to Lua, and then detect and translate it back into a Python exception when it turns out that Lua didn't handle it and passed it back to Lupa (i.e. when there is no pcall() wrapping it).

I wouldn't call it "intended behaviour", but I'm not sure it's worth changing.

[kmike]

I see, thanks for explanation.

I worked around it by wrapping Python functions to a decorator that converts Python exceptions to False, repr(exception) and return values to True, result; in Lua these Python functions are wrapped in a Lua decorator which raises an error if status is False (these errors can be catched by pcall) and returns the result otherwise.

This means that Python exceptions are not propagated back to Python as-is when they are not handled, they are converted to LuaErrors. For my use cases it is even better because for these new errors error messages include line numbers. Maybe in future I'll attach original Python exception objects to errors as userdata; for the current use cases repr is enough.

It seems both behaviours have their use cases: raising original errors and raising wrapped errors, even in a single runtime.

At the moment I don't have plans to work on it further because an existing solution works well enough, but supporting something like this directly in lupa could help to clean up some of the messy code. Creating an existing workaround was quite tricky, and it has edge cases with coroutines (Python decorator doesn't fix values sent using coroutine.send).