Kristin Cowalcijk

`isnumber` is defined in `ctype.h` so I renamed it to is_number to avoid name clashing, however `fmemopen` was undefined so I got a link error; Then I found an implementation...

Well I got the fix, I need to call `GC_init()` before any memory allocations calls (`GC_MALLOC`), now the test_repl worked but test_vm still suffers from assertion failure.

I have filed https://github.com/apache/arrow-java/pull/705 for fixing this.

Apache Arrow Java has made a new release with the fix: https://github.com/apache/arrow-java/releases/tag/v18.3.0. We can bump the version of arrow-java to close this issue.

> Please ping me when ready for review again. Thanks! @wgtmac I have added ColumnIndex and covering support. Please help review this PR when you have time, thank you.

> I just finished reviewing it for the 2nd pass. Thanks for the great work! > > My main concern is the difference with Java PoC, which generates min/max values...

I've copied the [tests on my branch](https://github.com/Kontinuation/datafusion-comet/blob/fix-shuffle-write-over-reservation/native/core/src/execution/datafusion/shuffle_writer.rs#L1445-L1509) to this PR and the test hangs: ``` running 6 tests test execution::datafusion::shuffle_writer::test::test_slot_size ... ok test execution::datafusion::shuffle_writer::test::test_pmod ... ok test execution::datafusion::shuffle_writer::test::test_insert_larger_batch ... ok...

Yes, the initial memory management [proposal](https://docs.google.com/document/d/1BT5HH-2sKq-Jxo51PNE6l9NNd_F-FyyYcyC3SKTnkIA/edit) and [implementation](https://github.com/apache/datafusion/pull/1526) did support cooperative spilling. However, a later [simplification](https://github.com/apache/datafusion/pull/4522) removed that feature. I believe using a shared `FairSpillPool` for comet operators within the...

I've [experimented](https://github.com/Kontinuation/datafusion-comet/tree/switch-to-fair-spill-pool) with approach 2 (per-task FairSpillPool) and it worked pretty well. I've also tried out per-worker FairSpillPool but it worked poorly, I'm still trying to figure out why. I've...

Thank you so much for declaring the recommended setup! It's given us a great direction, especially towards enhancing the memory manager in "unified" mode. Currently, I think the absence of...