Andy Grove
Andy Grove
failures are due to `return_field_from_args should be called instead`
> Is it possible to have arrow-rs 55.1.0 in datafusion 48.0.0.? A performance improvement went in for int8/int16 which was as a result of the unsigned int issues we raised....
Thanks for looking at this @dharanad. You are correct, Spark does not support this. We should probably remove `RightSemi` from the `operator.proto` file to avoid confusion: ```protobuf enum JoinType {...
Here is some more info. I am running with this config: ``` --conf spark.executor.cores=8 \ --conf spark.executor.memory=16G \ --conf spark.memory.offHeap.enabled=true \ --conf spark.memory.offHeap.size=24g \ ``` The memory issues start to...
more debug logging, which now more clearly shows what is happening: ``` 25/09/26 15:50:30 INFO CometTaskMemoryManager: Task 1561 requested 734496 bytes 25/09/26 15:50:30 INFO core/src/execution/memory_pools/unified_pool.rs: Task 1561 shrink() release_to_spark(734496) 25/09/26...
DataFusion operators are attempting to release more memory than acquired. We could choose to just ignore attempts to make pools negative, but I'd rather track down the root cause because...