Matthew Rocklin

The min/max values are available through the [deltalake.DeltaTable.get_add_actions](https://delta-io.github.io/delta-rs/usage/examining-table/#current-add-actions) API mentioned in this issue: https://github.com/delta-io/delta-rs/issues/2233#issuecomment-1971991963

I'm going to reopen this for now. I think that we could probably do better by looking at the data coming out of the `get_add_actions` function.

I don't know enough about this specific ML metric, but maybe if we can translate the problem to something more general then I can be of use. For example, If...

My gut reaction to this is skepticism. Large graphs are less of a problem than optimal scheduling. This approach helps with large graphs at the cost of optimal scheduling. I...

My guess is that query optimization will make more users happy than losing annotations will make users sad. Because of this, I'm in favor of merging in query optimization even...

Yup, I don't mind seeing improvements to annotations UX. However, I don't think that the release of query planning should be slowed down by this. I agree with scope, I...

I do not immediately hate this plan On Tue, May 10, 2022 at 3:32 PM Richard (Rick) Zamora < ***@***.***> wrote: > Others should feel free to reject this idea...

Same with `gc.collect()` ``` Filename: memory_issue.py Line # Mem usage Increment Occurences Line Contents ============================================================ 8 98.180 MiB 98.180 MiB 1 @profile 9 def test(): 10 250.863 MiB 152.684 MiB...

I believe that I have run this with MALLOC_TRIM_THRESHOLD_=0 already and saw the same results, but I should verify On Tue, Aug 24, 2021 at 5:57 PM Gabe Joseph ***@***.***>...

Yes, same result on my linux/ubuntu machine running mambafoge.