apache
add an "overrides" annotation
fix(presto): create column info - return native type
SUMMARY
Current Presto db_engine_spec get_columns failes because _create_column_info returned a stringifyied type instead of native type. This has prevented the creation of a new Presto dataset. See issue #25962
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Before:
After:
TESTING INSTRUCTIONS
run: pytest -v tests/unit_tests/db_engine_specs/test_presto.py
output: ============================================================================================================ test session starts ============================================================================================================ platform darwin -- Python 3.9.6, pytest-8.2.0, pluggy-1.5.0 -- ***/venv/bin/python3 cachedir: .pytest_cache rootdir: *** configfile: pytest.ini plugins: mock-3.14.0 collected 23 items
tests/unit_tests/db_engine_specs/test_presto.py::test_convert_dttm[VARCHAR-dttm0-None] PASSED [ 4%] tests/unit_tests/db_engine_specs/test_presto.py::test_convert_dttm[DATE-dttm1-DATE '2022-01-01'] PASSED [ 8%] tests/unit_tests/db_engine_specs/test_presto.py::test_convert_dttm[TIMESTAMP-dttm2-TIMESTAMP '2022-01-01 01:23:45.600000'] PASSED [ 13%] tests/unit_tests/db_engine_specs/test_presto.py::test_convert_dttm[TIMESTAMP WITH TIME ZONE-dttm3-TIMESTAMP '2022-01-01 01:23:45.600000'] PASSED [ 17%] tests/unit_tests/db_engine_specs/test_presto.py::test_convert_dttm[TIMESTAMP WITH TIME ZONE-dttm4-TIMESTAMP '2022-01-01 01:23:45.600000+00:00'] PASSED [ 21%] tests/unit_tests/db_engine_specs/test_presto.py::test_get_column_spec[varchar(255)-VARCHAR-attrs0-GenericDataType.STRING-False] PASSED [ 26%] tests/unit_tests/db_engine_specs/test_presto.py::test_get_column_spec[varchar-String-None-GenericDataType.STRING-False] PASSED [ 30%] tests/unit_tests/db_engine_specs/test_presto.py::test_get_column_spec[char(255)-CHAR-attrs2-GenericDataType.STRING-False] PASSED [ 34%] tests/unit_tests/db_engine_specs/test_presto.py::test_get_column_spec[char-String-None-GenericDataType.STRING-False] PASSED [ 39%] tests/unit_tests/db_engine_specs/test_presto.py::test_get_column_spec[integer-Integer-None-GenericDataType.NUMERIC-False] PASSED [ 43%] tests/unit_tests/db_engine_specs/test_presto.py::test_get_column_spec[time-Time-None-GenericDataType.TEMPORAL-True] PASSED [ 47%] tests/unit_tests/db_engine_specs/test_presto.py::test_get_column_spec[timestamp-TIMESTAMP-None-GenericDataType.TEMPORAL-True] PASSED [ 52%] tests/unit_tests/db_engine_specs/test_presto.py::test_create_column_info[columnVarchar-VARCHAR] PASSED [ 56%] tests/unit_tests/db_engine_specs/test_presto.py::test_create_column_info[columnString-String] PASSED [ 60%] tests/unit_tests/db_engine_specs/test_presto.py::test_create_column_info[columnChar-CHAR] PASSED [ 65%] tests/unit_tests/db_engine_specs/test_presto.py::test_create_column_info[columnInteger-Integer] PASSED [ 69%] tests/unit_tests/db_engine_specs/test_presto.py::test_create_column_info[columnTime-Time] PASSED [ 73%] tests/unit_tests/db_engine_specs/test_presto.py::test_create_column_info[columnTimestamp-TIMESTAMP] PASSED [ 78%] tests/unit_tests/db_engine_specs/test_presto.py::test_get_schema_from_engine_params PASSED [ 82%] tests/unit_tests/db_engine_specs/test_presto.py::test_where_latest_partition[column_type0-2023-05-01-DATE '2023-05-01'] PASSED [ 86%] tests/unit_tests/db_engine_specs/test_presto.py::test_where_latest_partition[column_type1-2023-05-01-TIMESTAMP '2023-05-01'] PASSED [ 91%] tests/unit_tests/db_engine_specs/test_presto.py::test_where_latest_partition[column_type2-2023-05-01-'2023-05-01'] PASSED [ 95%] tests/unit_tests/db_engine_specs/test_presto.py::test_where_latest_partition[column_type3-1234-1234] PASSED [100%]
============================================================================================================ 23 passed in 0.68s =============================================================================================================
ADDITIONAL INFORMATION
- [x ] Has associated issue: Fixes #25962
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in SIP-59)
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
@eyalsh99 thanks for the fix. I was wondering whether you did a git blame to try to decipher why we were quoting the type in the past? I agree it seems atypical, but there likely was some pseudo valid reason.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 67.39%. Comparing base (
c975f97) to head (38033f2). Report is 31 commits behind head on master.
Additional details and impacted files
@@ Coverage Diff @@
## master #28305 +/- ##
==========================================
+ Coverage 60.48% 67.39% +6.90%
==========================================
Files 1931 1933 +2
Lines 76236 76494 +258
Branches 8568 8568
==========================================
+ Hits 46114 51553 +5439
+ Misses 28017 22836 -5181
Partials 2105 2105
| Flag | Coverage Δ | |
|---|---|---|
| hive | ? |
|
| mysql | 77.52% <ø> (?) |
|
| postgres | 77.65% <ø> (?) |
|
| presto | ? |
|
| python | 77.78% <ø> (+14.29%) |
:arrow_up: |
| sqlite | 77.11% <ø> (?) |
|
| unit | ? |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
