spikely icon indicating copy to clipboard operation
spikely copied to clipboard

Asynchronously run a pipeline multiple times.

Open Shawn-Guo-CN opened this issue 6 years ago • 1 comments

Under the current version (0.5.2), users could run a same pipeline with same parameters multiple times. This would raise a potential problem, i.e. I/O conflicts of same tmp folders. Further, users wouldn't realise this problem, as the error wound only appear in stdout and never appears in our main window.

Shawn-Guo-CN avatar Oct 02 '19 13:10 Shawn-Guo-CN

I agree that this is a concern. @colehurwitz is also thinking about this.

It seems to me that on a per element basis there are three possibilities:

  1. Fully Concurrent: The element fully supports multi-instance concurrent execution. A bandpass filtering preprocessor, for example, should be able to run in multiple concurrent pipelines without any risk of failure.
  2. Parameter Concurrent: The element supports multi-instance concurrency, but requires a per-instance change to its parameters. A sorter, for example, that has a save path may be able to operate concurrently with another instance of the same sorter if and only if the save path is set per instance.
  3. Not Concurrent: The element has no support for multi-instance concurrency.

I'm tempted to solve this by adding a method to SpikeElement that would allow the app to query an element instance to determine its concurrency requirements, but I'm open to other suggestions.

rogerhurwitz avatar Oct 02 '19 14:10 rogerhurwitz