Josh Whelchel

Results 20 comments of Josh Whelchel

@tkaitchuck Perhaps a bit unrelated, but was there any thought put forward about pulling the `ShufflePipeline` out in such a way that a java module in a dedicated taskqueue could...

@aozarov Whoa! That flew right under my radar! As someone who has never launched a java appengine module before, is there any guide for deploying something like this an integrating...

Is this related to the 60s timeout on loading JSON responses for those large nesting pipelines? (We've thought about pointing our pipeline UI to a backend instance for this reason)

@tkaitchuck That statement seems entirely dependent on the amount of work being performed for each datum. I can confirm large 1,000+ shard jobs running, but @tkaitchuck is very right, that...

Good call using `use_barrier_indexes` for now - this is a potentially tricky one. If we move to fix this, it _may_ be surprising behavior for others. We use namespaces as...

Obviously, longterm: you don't want to kick off one pipeline per namespace permanently, you lose the major benefits of pipeline fan-out abortion and success if you aren't yielding child pipelines....

I have seen this issue too, where the pipeline stays in the Finalizing status, when it should have otherwise `aborted`, but I am not certain that this is the root...

@ymohii Can you share the root pipeline code, specifically if you've got something going on in the `finalized` method? - Obfuscate any business logic as necessary. I've seen this problem...

https://github.com/Khan/appengine-mapreduce has fixed the problem in their implementation by providing truncated values. I've implemented a fix for this as well, and the only catch was to ensure that we aren't...

I've started taking a look at bringing the truncation & `&depth=` UI features in. Kahn took their own approach to the consistency problem resolved here with `_BarrierIndex` (see their `_BarrierListener`)....