dbt-core icon indicating copy to clipboard operation
dbt-core copied to clipboard

[#9791] Fix datetime.datetime.utcnow() is deprecated as of Python 3.12

Open slothkong opened this issue 1 year ago • 5 comments

Resolves #9791

Problem

Any usage of datetime.datetime.utcnow() throws a deprecation warning in Python 3.12 when unittests are executed.

Solution

To get some results firsts, I just want to help with:

  1. the execution of your recommendations @dbeatty10, e.g. datetime.utcnow() -> datetime.now(timezone.utc).replace(tzinfo=None)
  2. bringing a bit of homogeneity to the ways datetime and its classes are being invoked, such that future development would require less effort

Code review checklist

  • [ ] Confirm that there are no breaking changes to dbt/artifacts
    • Otherwise, remove the artifact_minor_upgrade label. See #9839 (comment) for context.

Checklist

  • [X] I have read the contributing guide and understand what's expected of me
  • [X] I have run this code in development and it appears to resolve the stated issue
  • [X] This PR includes tests, or tests are not required/relevant for this PR
  • [X] This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

Additional Context

  • I successfully ran unit and integration tests on Python 3.12.0, Ubuntu 22.04
  • For good measure, I also built some models with both dbt-postgres and dbt-bigguery (1.8.0b1), then browsed over the resulting manifest.json files and verified that time references were accurate.
  • Currently, the only deprecation warnings raised during unittesting are related to dbt-common and logbook dependencies:
tests/unit/task/test_base.py: 2 warnings
  /path/to/my/dbt-core/.tox/py/lib/python3.12/site-packages/dbt_common/events/helpers.py:12: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    ts = datetime.utcnow()
tests/unit/task/test_base.py: 2 warnings
  /path/to/my/dbt-core/.tox/py/lib/python3.12/site-packages/dbt_common/events/logger.py:141: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    ts: str = datetime.utcnow().strftime("%H:%M:%S")
tests/unit/test_events.py::TestAdapterLogger::test_set_adapter_dependency_log_level
  /path/to/my/dbt-core/.tox/py/lib/python3.12/site-packages/logbook/compat.py:143: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
    return datetime.utcfromtimestamp(timestamp)

slothkong avatar Apr 01 '24 11:04 slothkong

Codecov Report

Attention: Patch coverage is 76.47059% with 8 lines in your changes missing coverage. Please review.

Project coverage is 88.05%. Comparing base (71f3519) to head (205a69e). Report is 203 commits behind head on main.

Files with missing lines Patch % Lines
core/dbt/contracts/sql.py 0.00% 3 Missing :warning:
core/dbt/parser/manifest.py 71.42% 2 Missing :warning:
core/dbt/logger.py 50.00% 1 Missing :warning:
core/dbt/task/sql.py 0.00% 1 Missing :warning:
core/dbt/utils.py 66.66% 1 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9839      +/-   ##
==========================================
- Coverage   88.12%   88.05%   -0.08%     
==========================================
  Files         178      178              
  Lines       22458    22458              
==========================================
- Hits        19792    19775      -17     
- Misses       2666     2683      +17     
Flag Coverage Δ
integration 85.41% <76.47%> (-0.17%) :arrow_down:
unit 61.78% <47.05%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Apr 01 '24 11:04 codecov[bot]

Judging by what is noted in the README, about changing code of dbt artifacts, it should be acceptable to add the artifact_minor_upgrade tag to this PR, no?

Changes made to core/dbt/artifacts/schemas/base.py, core/dbt/artifacts/schemas/results.py and core/dbt/artifacts/schemas/run/v5/run.py do not fall into the braking changes category.

Just for the record, the returned type of datetime.utcnow() is equivalent to that of datetime.now(timezone.utc).replace(tzinfo=None) (e.g. <class 'datetime.datetime'>), and both methods work across python versions, from 3.12 all the way to 3.8.

slothkong avatar Apr 01 '24 14:04 slothkong

I will try to submit an additional commit this week, to raise the patch coverage, specifically for core/dbt/contracts/sql.py~

slothkong avatar Apr 14 '24 16:04 slothkong

Judging by what is noted in the README, about changing code of dbt artifacts, it should be acceptable to add the artifact_minor_upgrade tag to this PR, no?

@slothkong Since I'm not aware of any breaking changes, I've added the artifact_minor_upgrade label.

But to cover our bases, I've added this checklist item for the code review:

image

dbeatty10 avatar Apr 15 '24 16:04 dbeatty10