Noah D. Brenowitz
Noah D. Brenowitz
I just wanted to comment that this would be a very useful feature for me. Currently, I am automating my computational workflows using Snakemake and make, which then calls python...
I have glanced at that page, but it seemed to me that that is in-memory caching only. I basically want to be able to manually specify the nodes of a...
Okay, I might start another issue, but the memoization syntax that @alimanfoo proposes above would also probably handle my use-case pretty nicely.
Yah, I would be more than happy with B = cache.memoize(really_expensive_or_interesting_computation)(A)
I agree that could be a good collaboration point. How do you see this going? I could see xarray-beam depending on xpartition, the shared functionality being up-streamed to xarray, or...
I think we probably have enough experience that we could make a decent design, so I wouldn't be too worried about needing to track xpartition too closely as a dependency....
I'm also not convinced this simple functionality would be more maintainable in xarray, although it could replace the map_blocks function, which I assume is not particularly easy to maintain...
I do see @rabernat's point about establishing APIs and then optimizing, but I'm not sure many will use this project without convincing performance benchmarks. The competition for `xbatcher` is simple...
> You seem to be implying that we should abandon the project entirely. Is that the correct interpretation of your comment? Not exactly...just pointing out that "use a for-loop" might...
@rabernat recent post inspired me to check out this issue. What would this issue entail now that `__array_function__` is in numpy? Is there some reason this is more complicated than...