nprintml icon indicating copy to clipboard operation
nprintml copied to clipboard

background tasks

Open jesteria opened this issue 3 years ago • 2 comments

The pipeline might be sped up by performing in the background sub-tasks which are necessary for completion but not for the pipeline itself.

For example, the user may have requested that nPrint data and/or features are saved to disk; however, the pipeline needn't necessarily wait for these to be serialized and written to disk before continuing to the next step in the pipeline.

Instead, such work might be performed by a background thread/process (off of a standard queue), while the pipeline proceeds.

jesteria avatar May 03 '21 22:05 jesteria

This should be considered and tested to ensure we're able to measurably improve performance this way and to ensure that whatever we do doesn't interfere with the performance of the pipeline itself.

jesteria avatar May 03 '21 22:05 jesteria

Agreed on the testing -- this seems significantly less important given the results in #55, though as the files grow in size, it could become more important to consider this.

JordanHolland avatar May 04 '21 13:05 JordanHolland