superset icon indicating copy to clipboard operation
superset copied to clipboard

feat: add support for catalogs

Open betodealmeida opened this issue 9 months ago • 2 comments

SUMMARY

Add support for catalogs to:

  • Presto
  • Trino
  • Snowflake
  • BigQuery

We already have support for Postgres and Databricks.

@john-bodley I'm worried about the DB migration here and how it will affect AirBnB. Can you take a look? The migration needs to:

  • Read the default catalog for each of these 4 DBs and update the catalog column in a few models (SqlaTable, Query, SavedQuery, TabState, and TableSchema).
  • Update the schema_perm column for any SqlaTable and Slice models in these databases.
  • Rename existing permissions of type schema_perm.

I'm happy to hold back on this PR until we know it's safe to merge to master.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • [X] Has associated issue: https://github.com/apache/superset/issues/22862
  • [ ] Required feature flags:
  • [ ] Changes UI
  • [X] Includes DB Migration (follow approval process in SIP-59)
    • [X] Migration is atomic, supports rollback & is backwards-compatible
    • [ ] Confirm DB migration upgrade and downgrade tested
    • [ ] Runtime estimates and downtime expectations provided
  • [ ] Introduces new feature or API
  • [ ] Removes existing feature or API

betodealmeida avatar May 09 '24 22:05 betodealmeida

Thanks @betodealmeida for the ping. I think we (Airbnb) are ok with this. We will likely just skip the catalog_access as we don't use the FAB permission model for Superset entities.

john-bodley avatar May 10 '24 17:05 john-bodley

Looking forward to this! We rely on using the superset interface with trino and many other databases. And full catalog with all connected sources will be meaningful to our data team 🙌❤️😄

stephenlb avatar May 10 '24 18:05 stephenlb