parallel icon indicating copy to clipboard operation
parallel copied to clipboard

Add ForkContext

Open trowski opened this issue 9 months ago • 9 comments

This adds a Context implementation which uses pcntl_fork to create another process. This is a strategy which we abandoned some time ago due to the issues with copying parent context into a child, however there are particular case where that behavior can be advantageous.

This context is NOT added to DefaultContextFactory, and in fact has warnings against its use in the class docblock.

@danog Can you please check if this works for you in Psalm.

trowski avatar Mar 01 '25 15:03 trowski

This does not work in psalm, because runContext is queueing a callback onto the event loop, which causes an immediate The event loop is already running error when hitting EventLoop::run(); unwrapping the callback fixes the issue.

danog avatar Mar 01 '25 19:03 danog

Additionally, I added some extra error handling logic to handle segfaults in child processes (commonly caused by JIT usage in psalm), would love to see that in this impl as well: https://github.com/vimeo/psalm/blob/fc437245b15a7220de107bc191a3559fdbf94595/src/Psalm/Internal/Fork/ForkContext.php#L61

danog avatar Mar 01 '25 19:03 danog

checkExit should probably also be executed within the constructor in a finally block as well, as https://github.com/vimeo/psalm/issues/11310 is caused by a crash quite possibly before the initial IPC handshake is completed, haven't further looked into this yet...

danog avatar Mar 01 '25 19:03 danog

This does not work in psalm, because runContext is queueing a callback onto the event loop, which causes an immediate The event loop is already running error when hitting EventLoop::run(); unwrapping the callback fixes the issue.

My local test was running from {main}, so I missed that. Unfortunately running this new context through the usual test suite crashes and burns, of course because we're forking the test process.

I removed the event loop queuing from runContext and updated ForkContext with the recent changes you made. Give it another try when you get a chance.

trowski avatar Mar 01 '25 20:03 trowski

What is the use case you mentioned?

kelunik avatar Mar 02 '25 07:03 kelunik

What is the use case you mentioned?

@danog Can you please elaborate on how you're using this in Psalm?

trowski avatar Mar 02 '25 21:03 trowski

I'm using this in psalm to avoid serializing and sending tens of megabytes of data structures containing information about the codebase post-scanning, when starting the analysis phase.

danog avatar Mar 03 '25 09:03 danog

https://github.com/danog/parallel/tree/fork-context contains some more improvements and enables (most) unit tests, working OK in Psalm!

danog avatar Mar 07 '25 16:03 danog

Ping @trowski :)

danog avatar Mar 16 '25 17:03 danog