wren-lang
Reenterability
According to documentation, the VM is not reenterable:
While your foreign method is executing, the VM is completely suspended. No other fibers run until your foreign method returns. You should not try to resume the VM from within a foreign method by calling wrenCall() or wrenInterpret(). The VM is not re-entrant.
Is it still true? How is it supposed to be able to call ShowHeroDialog from wren and call wren again for dialog events? We cannot create another fiber and pass control to it on event?
It is still true yep. I'm working on a PR atm for this, I'm testing in a large use case. @mhermier has an older branch that is reentrant proving it feasible, but we're still pinning down some details for it like stack management.
Typically the suggestion is to defer your callbacks and call out instead from your main loop, but this isn't always feasible (physics simulation step callbacks, rendering callbacks mid frame). If they're async style like dialogs, http or the like - it works fine to queue them and process them all in one go.
I'm currently toying with an extra operator branch, that will be rejected by Bob because of speed, but has the benefits to explicit Bool conversions and null checking. After that I'll try to give a try a producing a small renetrency branch. Cheers
Le 30 nov. 2017 7:18 PM, "Sven Bergström" [email protected] a écrit :
It is still true yep. I'm working on a PR atm for this, I'm testing in a large use case. @mhermier https://github.com/mhermier has an older branch https://github.com/mhermier/wren/tree/reentrancy that is reentrant proving it feasible, but we're still pinning down some details for it like stack management.
Typically the suggestion is to defer your callbacks and call out instead from your main loop instead, but this isn't always feasible (physics simulation step callbacks, rendering callbacks mid frame). If they're async style like dialogs, http or the like it works fine to queue them and process them all in one go.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/munificent/wren/issues/487#issuecomment-348274715, or mute the thread https://github.com/notifications/unsubscribe-auth/AY7blRYoZqfjDyXW5S93CVoIPPWnoXwWks5s7vFXgaJpZM4QwxR- .
I have an idea: making wrenInterpret reentrant may be simpler than making the Wren VM reentrant.
Suppose we have a new kind of foreign method call that supports reentrancy? When called, the call could could stash its arguments somewhere, suspend the current fiber, and return, similar to what we do for I/O. Just before the wrenInterpret call exits, it could check if there is a pending foreign method call, and if so, run it. In this state, there wouldn't be any restriction on reentering Wren. So, we wouldn't have multiple VM's suspended on the C stack, but you could have multiple invocations of wrenInterpret on the stack.
There would be more call overhead compared to the current, non-reentrant foreign methods, but maybe it's not too bad?
It is quite easy, but not as easy as you think. There are 2 places in current code where there is something basic reentrancy as you discribe and it is bugged as hell for various reasons. So there is no extra cheap way to do it, and the proper way is a quite expensive due to API/design decision.
I don't find the places because they were mitigated with a patch which is not sufficient for reentrancy.
The issue is that when the reentrancy happens, there is a desync of some cached variables within the functions. Not it is a bad solution, but because of design decision, as soon as wrenEnsureSlots get called or some WrenFiber::frames get resized there is a chance that vm state is broken because it requires more checkings.
And because of these design decision, the API don't allow you to create a new context from a foreign call. So we could have mitigated the issue by creating a separated execution context but it is not possible. The API only allow to create a complete new WrenVM.
So the only abuse you can do is trying to call wrenEnsureSlot/wrenCall within a foreign call. While wrenCall protect you from doing it at runtime, wrenEnsureSlot does not. But the problematic things is that there are some situations where you need to increase the slot stack size to process some data.
So we are currently moistly doomed because of the API.
Any more updates on this? Is it stuck because it would involve an incompatible API change, or is backwards compatibility not a concern?
The more obvious reason is that it is beyond project goals, unless some needs ultimately emerge.
I see. I would very much like to get this working myself. As someone who's not very familiar with the Wren codebase, what would it take for me to make this happen (perferably in a way that I can contribute back to upstream?)
I wonder if the Lua VM is totally re-entrant the Wren VM can't be the same. Not having the VM re-entrant does enforce some pretty high limitations for making certain methods/functions as they are written in Wren instead of C itself. I'm not entirely sure how Lua achieves, I'm yet to understand how. This issue definitely needs more attention to be honest.
The issue hasn't been ignored fwiw, I've been investigating a path, it's a bit tricky to test at scale and measure perf impacts etc. But it's worth noting it's actively pursued with some recent success ✨
@ruby0x1 FWIW this is something I am interested in as well. Basically I cannot call a wren callback from a foreign method. I need this for .then style chaining for a Task system. This imposes a severe limitation compared to Chaiscript.
On the positive side, the fibers support and the ergonomics of the language are really good so far and I am very happy with it. But I am missing the callbacks part :(
@datatypevoid why the negative vote? 🤔
@ruby0x1 is there ongoing work on this? Is there any way I could help maybe? I am hitting a blocker with this to the point that I am considering migrating to Squirrel :(