seafowl
seafowl copied to clipboard
Expose some DataFusion runtime execution settings in the config
Done in https://github.com/splitgraph/seafowl/pull/71:
~~- MemoryManagerConfig
: max runtime memory usage for plan execution (rough, since it doesn't track basic process data structures): https://github.com/apache/arrow-datafusion/blob/master/datafusion/core/src/execution/memory_manager.rs#L35-L55 / https://github.com/apache/arrow-datafusion/blob/master/datafusion/core/src/execution/runtime_env.rs#L141-L145~~
~~- DiskManagerConfig
: using the OS temp directory / other path to spill data to disk during execution: https://github.com/apache/arrow-datafusion/blob/master/datafusion/core/src/execution/disk_manager.rs#L50-L65 / https://github.com/apache/arrow-datafusion/blob/master/datafusion/core/src/execution/runtime_env.rs#L135-L139~~
Misc:
- Planner/executor config options from https://github.com/apache/arrow-datafusion/blob/master/datafusion/core/src/config.rs and https://github.com/apache/arrow-datafusion/blob/master/datafusion/core/src/execution/context.rs#L1038-L1060