Consider using Citus extension for columnar (compressed) data

[burggraf]

I'm investigating the use of the Citus PostgreSQL Extension to bring additional functionality to PostgreSQL. Mainly I'm interested in columnar storage, which, in my initial test, compresses data by around 32x.

All the features made available by Citus include:

• Distributed tables are sharded across a cluster of PostgreSQL nodes to combine their CPU, memory, storage and I/O capacity.
• References tables are replicated to all nodes for joins and foreign keys from distributed tables and maximum read performance.
• Distributed query engine routes and parallelizes SELECT, DML, and other operations on distributed tables across the cluster.
• Columnar storage compresses data, speeds up scans, and supports fast projections, both on regular and distributed tables.

[burggraf]

Installation: Citus provides a docker container install for testing, and an easy apt-get installation for Ubuntu, however there's no current package for ARM available, so for my Supabase instance I had to compile it from source.

After installation, Postgres needed a restart, then the extension was easily loaded with CREATE EXTENSION citus;. The columnar storage features work great. I'll do some benchmarks on them soon.

[elemen3]

How is it going @burggraf ?

[dragarcia]

Heya @burggraf

• I've created a new thread on https://github.com/supabase/supabase/discussions/679
  • https://github.com/supabase/supabase/discussions/679#discussioncomment-3731774
• Let's continue the discussion there to have a single source for updates as well as a consolidated way of gauging the interest of the community.