mpipe
mpipe copied to clipboard
vmlaker
Python API for writing multiprocessing pipelines
Addition of an AsyncPipeline than consumes a normal pipeline, and gerator as a data source, and provides and async generator for the pipeline output. Addresses #17
我想问下。我该怎么修改，才能完成实时的获取到最新的数据，如果我下一个进程处理不过来的话，可以将旧的数据清除掉，保障每次获取的都是最新的数据，而不是一直将数据塞入缓冲区，最后导致内存撑爆。 @vmlaker
I'm using `mpipe` for computer vision processing from camera frames, it's exactly what I need, but the `put()` `get()` api is nonideal, and I can't figure out a way to...
On OS without `fork` support, `wclass` in `OrderedStage.__init__` is required to be serializable, that's a problem. Otherwise an exception will be raised: > AttributeError: Can't pickle local object 'OrderedStage.__init__..wclass'
Pipeline.get return only one result from its output_stages . I can not catch the design, or a bug ? def get(self, timeout=None): """Return result from the pipeline.""" result = None...
Add the tox virtualenv management into the flow to be able to test for different Python versions.
I'm trying this for the first time, and the simple example on the main page does not work: http://vmlaker.github.io/mpipe/ Tried on python 3.4.5 and 3.6.1 Windows 7 Anaconda 4.4.0 64-bit...
I'm seeing hangs using `OrderedWorker` if the payload used in `put` is longer than a certain size. Here is an example that will randomly hang on my machine if a...
Add to documentation an example of how to distribute an embarrasingly parallel problem to mulitple cores. Consider a cascade of concentric for-loops, marshalling the outer for-loop to workers.
I've tried a pipeline like this: stage1 --> stage2 --> stage3 stage1 --> stage3 That is, stage3 receives tasks from both stage1 and stage2. I've get some wired results. does...