pg_analytics
pg_analytics copied to clipboard
test: Support for Multi-Level Partition Tables
What
Implements a demonstration test for multi-level partition tables, addressing issue #56.
Why
This demo showcases the pg_analytics
extension's capability to support multi-level partitioned tables. The implementation organizes data hierarchically, enabling efficient access to context-relevant information.
How
- Generates a simulated Auto Sales dataset and creates a local Parquet data source.
- Utilizes DataFusion to load sales data from the Parquet file into a DataFrame.
- Partitions the data and uploads partitions to an S3 bucket.
- Configures necessary tables and
pg_analytics
Foreign Data Wrapper (FDW) in PostgreSQL using S3 data. - Executes analytics test queries on the multi-level partitioned table.
Tests
To run the demonstration test:
RUST_LOG=info \
cargo test \
--test \
test_mlp_auto_sales \
-- \
--nocapture
Test traces are available in the attached log file :: https://gist.githubusercontent.com/shamb0/2ed909ac9604c610af1d7fa0e87f9a82/raw/02a4203cdc1d675181d9f9700c578c81405becdb/wk2434-pg_analytics-mlp-demo.md