superset icon indicating copy to clipboard operation
superset copied to clipboard

refactor: pass all properties to validate_parameters

Open eschutho opened this issue 2 years ago • 3 comments

Some databases require values that aren't in the sqlalchemy uri in order to connect. I'm passing in all properties from the validate_params api into the db_engine_spec for validation, instead of just the parameters so that we can continue to keep the client side api structure as similar as possible to the db database model and not have to add the same values in both the root db payload and parameters in order to do validation. This is some pre-work for adding in the databricks form that requires http_path from extras for connection.

Hopefully this change will help clarify the difference between parameters, and when something is a parameter in the db model and when it's not.

I'm also adding the driver to the db model, because as per @betodealmeida's change in https://github.com/apache/superset/pull/21171 if an engine can have multiple drivers, like databricks, we're going to need to pass that value in from the client side when creating/editing a db in order for us to find the correct db_engine_spec.

TESTING INSTRUCTIONS

Database creation with the new forms should work as expected.

ADDITIONAL INFORMATION

  • [ ] Has associated issue:
  • [ ] Required feature flags:
  • [ ] Changes UI
  • [ ] Includes DB Migration (follow approval process in SIP-59)
    • [ ] Migration is atomic, supports rollback & is backwards-compatible
    • [ ] Confirm DB migration upgrade and downgrade tested
    • [ ] Runtime estimates and downtime expectations provided
  • [ ] Introduces new feature or API
  • [ ] Removes existing feature or API

eschutho avatar Sep 15 '22 22:09 eschutho

Codecov Report

Merging #21487 (907e091) into master (4d0c2ba) will decrease coverage by 1.64%. The diff coverage is 90.00%.

:exclamation: Current head 907e091 differs from pull request most recent head aaeb7a5. Consider uploading reports for the commit aaeb7a5 to get more accurate results

@@            Coverage Diff             @@
##           master   #21487      +/-   ##
==========================================
- Coverage   66.79%   65.15%   -1.65%     
==========================================
  Files        1799     1791       -8     
  Lines       68829    68587     -242     
  Branches     7314     7320       +6     
==========================================
- Hits        45976    44689    -1287     
- Misses      20974    22008    +1034     
- Partials     1879     1890      +11     
Flag Coverage Δ
hive ?
mysql 78.20% <90.00%> (+<0.01%) :arrow_up:
postgres 78.27% <90.00%> (+<0.01%) :arrow_up:
presto ?
python 78.40% <90.00%> (-2.96%) :arrow_down:
sqlite 76.75% <75.00%> (?)
unit ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/db_engine_specs/snowflake.py 48.34% <50.00%> (-14.75%) :arrow_down:
superset/db_engine_specs/gsheets.py 46.51% <66.66%> (-29.07%) :arrow_down:
superset/databases/api.py 95.52% <100.00%> (+0.02%) :arrow_up:
superset/databases/commands/validate.py 77.58% <100.00%> (ø)
superset/db_engine_specs/base.py 81.46% <100.00%> (-7.87%) :arrow_down:
superset/db_engine_specs/bigquery.py 73.29% <100.00%> (-8.76%) :arrow_down:
superset/models/core.py 89.71% <100.00%> (-0.15%) :arrow_down:
superset/tables/schemas.py 0.00% <0.00%> (-100.00%) :arrow_down:
superset/columns/schemas.py 0.00% <0.00%> (-100.00%) :arrow_down:
...-chart-table/src/DataTable/utils/useMountedMemo.ts 0.00% <0.00%> (-87.50%) :arrow_down:
... and 214 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov[bot] avatar Sep 16 '22 20:09 codecov[bot]

/testenv up

eschutho avatar Sep 19 '22 16:09 eschutho

@eschutho Ephemeral environment spinning up at http://34.219.255.3:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

github-actions[bot] avatar Sep 19 '22 16:09 github-actions[bot]

test the db connection form in ephemeral env, LGTM

jinghua-qa avatar Sep 28 '22 17:09 jinghua-qa

Ephemeral environment shutdown and build artifacts deleted.

github-actions[bot] avatar Oct 04 '22 00:10 github-actions[bot]