Support multi process in download

Open oreenlivnicode opened this issue 2 years ago • 1 comments

Currently, passing Config object arrives empty to the index_workflow_file function. This happens because processes use separate memory, we should pass the relevant config params to the function or use Manager.

Sep 19 '23 08:09 oreenlivnicode

When we handle https://github.com/CycodeLabs/raven/issues/78 we can move to this. Multi-processing will cause a rate-limit very quickly.

Oct 09 '23 17:10 elad-pticha