Florian Jetter

Thanks for your report. Before I get into this, what you call "compilation", i.e. the process of building the "low level graph" / initializing all tasks, is what we typically...

Apart from the materialization issues, your example is generating a graph of about 4MM tasks. This is larger than what we are typically dealing with and especially if you are...

> layers, it should be the case that in the example above, the first block can be computed without actually materializing the other (unnecessary for that block) parts of the...

@rjzamora maybe you got some insights here?

I actually don't have a very strong opinion here. I'm mildly concerned about confusion internally and about migration efforts (for users). If we believe this is worth it, I'm fine...

Sorry for the late reply. I can reproduce this. here is another reproducer (note that `jit` works) ```python from numba import guvectorize, int64, jit @jit(nopython=True, cache=True) def f(x, y): return...

Sorry, I believe the error I am reproducing with my example is actually a little different. My example only fails if the `cache` kwarg is `True`. The dask-examples version fails...

I think this is somewhat expected. The object `mapper` you are defining requires about 250kB in memory. When calling `map_blocks` this will generate _tasks_ internally that embed whatever arguments you're...

Typically it is expected that a task's definition is small and we are retaining the task until the computation is completed. Specifically, it is retained for as long a local...

In these situations we recommend one of two things 1. Ideally whatever data is in mapper can either be generated inside of a dask task or loaded from a remote...