[Epic] Support data source inserts

[scsmithr]

Currently everything is read only.

To start, we can create the execution plans. Follow on work may be needed for if we want to mark data sources read-only, but this will likely coincide with some sort of permissions system.

Tasks

• [x] #2010

Data Sources

• [x] #1883
• [x] #1889
• [x] #1961
• [ ] inserts into BigQuery
• [ ] Inserts into Snowflake
• [ ] Delta lake
• [ ] Iceberg
• [ ] Object store (we may be able to use datafusion's stuff for this)

Questions

• scope: share code between COPY TO and INSERT INTO and figure out how much work.
• identify limits for limits with number of values.
• [X] read only mode

[greyscaled]

I think we hold off on permissions, or just error on inserts if we have credentials that don't have insert permissions. Will want to document this behavior in glaredb docs.

[greyscaled]

@vrongmeal might be working on postgres soon

[greyscaled]

We're holding off on this for now for other priorities. We can revisit, and start with Pg as POC.

We will likely wait for a request.

[greyscaled]

moving from 0.6.0 to https://github.com/GlareDB/glaredb/milestone/25

[greyscaled]

Question (as a user/dogfooder): I am using a Postgres data source that I explicitly want to be read-only. It's critical that I do not accidentally perform any mutations (insert, update, delete, drop, alter etc etc): I simply want to only select from this data source.

How can I best achieve that? And how might we consider this problem longer-term?

[tychoish]

I think we should handle insert-into-object store (e.g. globbed file datasources) as part of a different epic.

@vrongmeal and I have been talking about doing delta/iceberg as part of the current push on table-like data formats. in the near term.

This leaves bigquery and snowflake, mostly. I'm not sure exactly what the right answer is for bigquery (aside from just writing to files in GCS?) and snowflake, which we could probably do relatively soon.

Otherwise we're continuing to make progress here.

[tychoish]

Just for an update on scope:

• I believe that inserts into object store (e.g. parquet,csv,bson,json) data sources should be handled in a separate epic, as they depend on some catalog changes to track prefix of a the table in the catalog.
• We're working on a seperate epic for table-format support (delta/iceberg) [lance already has full insert/write support].

These things shouldn't block completion of this epic, which means the remaining scope on this ticket is:

• cassandra
• clickhouse
• snowflake, bigquery