Flowise icon indicating copy to clipboard operation
Flowise copied to clipboard

Feature/Add child processes

Open HenryHengZJ opened this issue 2 years ago • 0 comments

Context: Allow users to run multiple API requests in parallel. https://python.langchain.com/en/latest/modules/agents/agent_executors/examples/async_agent.html

As Flowise is written in NodeJS, and using async await to run prediction, it is equivalent to Python asyncio. So API requests are run in parallel by default. Read more here

To further increase performance, we can use child process from NodeJS to spawn multiple processes, with each process running its own prediction. Read more here

By default, it will use NodeJS main thread to run prediction. To enable child process mode, go to .env file in packages/server and add this:

EXECUTION_MODE=child

You can also specify timeout to stop the child process:

EXECUTION_TIMEOUT=5000

HenryHengZJ avatar May 15 '23 22:05 HenryHengZJ