Florian Jetter

The problem you're facing here is that `nothing.compute` is a bound method, i.e. that callable keeps a reference to `Nothing` which keeps a reference to your `DataFrame` alive. Apart from...

So instead, you should try to do it like... ```python class Nothing: def __init__(self, x): self.x = x @staticmethod def compute(x, df): # do nothing return None ... out =...

to get the memory footprint down, it should be the delayed version which should just work. However, I noticed a bug that is not treating the delayed object properly.

There is one more option but that requires access to the client which often makes code more complex ```python x = client.submit(create_x) out = ddf.map_partitions(Nothing.compute, x, meta=pd.Series([None])) ``` This does...

I got a fix for the delayed thing up https://github.com/dask/dask/pull/11907 I suspect we'll cut another release today/tomorrow since there were a couple of regressions in the release earlier this week

[preliminary benchmark results](https://github.com/coiled/benchmarks/actions/runs/10161188571) are... interesting... I'll have to double check  FWIW in micro benchmarks the conversion of the old to new graph layout was up to 50% slower than...

In case anybody reviews. Just had a conversation with Hendrik and here are a couple of expected concerns that were raised and discussed - Sequence and Dict of task are...

> Wouldn't we want to do https://github.com/dask/distributed/pull/8797/files#r1731514684 client side to reduce the upload size? That would force materialization client side as well... which may be ok... I have to think...

I added a commit that changed the signature to `Task(key, func, /, *args, **kwargs)` which makes internals sometimes a little more awkward but is much more intuitive to use, I...

FYI Progress is a little hampered. I'm running into the wildest, pretty much unrelated, problems. Just to name a couple - There was a thing in our tokenization code -...