ibis-project
feat(databricks): allow read_only option for Databricks backend
Description of changes
Allow data access through Databricks with only READ access on the table, by skipping memory table.
Issues closed
- Resolves #11598
Is there a way we can support local data (pandas dataframes, polars dataframes, etc) that doesn't require a user to have volume writing permissions?
Lots of useful features require memtables, so it'd be nice if we can make this work without this flag.
Hi @cpcloud,
Not sure what you mean by "support local table", it would be to replace the volume created by a local table instead I guess. If memtable is useful even when only reading tables (not writing or anything else).
Then a local path would lead to a local table validation/creation, without any CREATE VOLUME required.
I'll look into it! My only concern is the support of "local commands" if the table is local, but let's see
To check the actual code coverage I believe that ci-run-cloud tag would be needed.
Otherwise the idea is quite simple, if the flag is given and no memtable_volume is given, it will piggyback on the Polars backend for the memtable functionalities, and keep that table local.
Hi @cpcloud , do you mind having a look to see if this is on the right way or if I need to change the approach ?
Hi @cpcloud, do you mind having a look at this we would need this too for our company.
Hi @cpcloud just +1 the usefulness of this fix when you have a moment to review 🙏
