datahub icon indicating copy to clipboard operation
datahub copied to clipboard

refactor: Unify get_current_checkpoint_from_pipeline helper

Open libvarun opened this issue 3 weeks ago • 2 comments

Summary of changes:

  1. Refactor: Moved get_current_checkpoint_from_pipeline to metadata-ingestion/src/datahub/testing/state_helpers.py to allow sharing between test suites.
  2. Cleanup: Removed duplicate implementation from smoke-test/tests/test_stateful_ingestion.py and updated metadata-ingestion tests to use the shared helper.
  3. Verification: Ran smoke-test and relevant unit tests locally, and they passed.

No Deps changed.

libvarun avatar Dec 05 '25 11:12 libvarun

@libvarun , linting please

deepgarg760 avatar Dec 05 '25 13:12 deepgarg760

LGTM!!

deepgarg760 avatar Dec 05 '25 13:12 deepgarg760

@deepgarg760 Please check now. linting issues could be IDE warnings only - the actual project linting passes. Your IDE's Python interpreter isn't pointing to the virtual environment.

libvarun avatar Dec 18 '25 07:12 libvarun

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests.

:loudspeaker: Thoughts on this report? Let us know!

codecov[bot] avatar Dec 18 '25 07:12 codecov[bot]

Bundle Report

Changes will decrease total bundle size by 2.51MB (-8.02%) :arrow_down:. This is within the configured threshold :white_check_mark:

Detailed changes
Bundle name Size Change
datahub-react-web-esm 28.77MB -2.51MB (-8.02%) :arrow_down:

Affected Assets, Files, and Routes:

view changes for bundle: datahub-react-web-esm

Assets Changed:

Asset Name Size Change Total Size Change (%)
assets/index-*.js -103.28kB 19.14MB -0.54%
~~assets/signup-*.mp4~~ (Deleted) -2.26MB 0 bytes -100.0% :wastebasket:
~~assets/airflowlogo-*.png~~ (Deleted) -37.61kB 0 bytes -100.0% :wastebasket:
~~assets/prestologo-*.png~~ (Deleted) -33.55kB 0 bytes -100.0% :wastebasket:
~~assets/sagemakerlogo-*.png~~ (Deleted) -17.81kB 0 bytes -100.0% :wastebasket:
~~assets/iceberglogo-*.png~~ (Deleted) -17.61kB 0 bytes -100.0% :wastebasket:
~~assets/s3logo-*.png~~ (Deleted) -16.96kB 0 bytes -100.0% :wastebasket:
~~assets/sparklogo-*.png~~ (Deleted) -16.15kB 0 bytes -100.0% :wastebasket:
~~assets/dagsterlogo-*.svg~~ (Deleted) -5.45kB 0 bytes -100.0% :wastebasket:

codecov[bot] avatar Dec 18 '25 09:12 codecov[bot]