eris
eris copied to clipboard
fnuecke
WIP: Persist coroutines yielded from hook
Inspired by wanting preemptive multitasking in OpenComputers. See my other repo for explanation of changes to debug.sethook() that let Lua hooks yield. After posting to OC forum, it turned out that Eris cannot persist coroutines yielded from hooks (and even has a special error message for them), so here's my attempt to fix that.
The main problem seems to be that normal yields never happen from Lua code, there's no opcode for that. At the same time, hook yields look like such opcode was the last one, there's no enclosing C call. I assumed that this case should be treated the same way as a Lua stack frame, since it is one, but thread->ci->func no longer points to the closure and has to be restorestack()ed. I've additionally stored the value of restored ci->func for unpersisting, since restorestack() relies on ci->extra, which will only be unpersisted after the loop. ~This breaks serialization format compatibility, but is fine for testing, and a workaround can be implemented later.~ Actually, that only happens when persisting hook-yielded coroutines, so there's nothing to break.
This PR is a WIP, intended to be cleaned up and squashed when it's ready. The code currently seems to work, and I've added a simple test for it. However, I suspect more thorough testing is required, and I have no idea what kind of tests to make. I'm not very familiar with Lua and/or Eris internals. Need help producing more tests!
Currently known problems:
- [ ] The "native"
gmatch()returns a closure that keeps its state in a userdata, and with yielding hooks it might have to be persisted. Maybe there are more similar cases. - [ ] There definitely is some memory corruption going on. (Un)Persisting enough times leads to a state where some calls to
lua_pushvfstring(L, ".%s", "key")reliably get "userdata" when pushing char* and fail, preventing any further persisting. - [ ] Not sure if it's related to the corruption, but I'm periodically getting "attempt to persist userdata" messages. They never are as reliable as the above crashes.
- [ ] Neither of the 2 "userdata" problems reproduces without the full game, as of yet, so I cannot even use valgrind to look for them...
Sorry for the late reply, had started typing out a reply but then I ran out of time and kinda forgot until now...
Fascinating! I'm afraid it's been a bit too long for me to just jump in and make informed comments, and I won't be having the time to dig into the code in the immediate future. But it'd be very cool to get this to work.
Though one thing I'd ask: could you wrap any "core" changes necessary for this to work in an #ifdef ERIS_PREEMPTIVE or something? Some people might want to disable this to stick closer to vanilla Lua. I'd also expect that to make porting easier because it's immediately visible what changed in core Lua and what changed in this patch.
Thanks a ton!
I kinda got distracted myself, not returning to it for at least a couple more weeks. The PR is more in the "make it work so I can use it to test the Lua part" state, I think I will finalize and squash it after I'm done with patching machine.lua and the scala code in OC. It probably still needs some work on serialization format, plus I had a weird segfault during serialization once and only have a vague suspicion why.
By the way, any idea how JNLua-internal functions might end up in the state when hook-yielding close to API calls, e.g. computer.uptime()? (calljavafunction, to be specific) Hooks should not trigger when native call is in progress, so I suspect that LUA stack is being handled incorrectly when persisting hook-yielded coroutine.
Could you please look into https://github.com/MovingBlocks/eris instead? We're migrating OpenComputers over to that, eventually.
Yes, whenever I get back to it. Life happened and I kinda forgot… Still haven't tracked down the abovementioned segfault and persisting problems.