Follow-up from "Resolve "Bug: `get_starting_timestamp()` and `get_starting_replication_key_value()` should be cached""
Migrated from GitLab: https://gitlab.com/meltano/sdk/-/issues/232
Originally created by @aaronsteers on 2021-10-07 19:01:48
The following discussion from !157 should be addressed:
-
[ ] @aaronsteers started a discussion: (+2 comments)
@edgarrmondragon - This is looking great. I just did another review and there's just one remaining change I'd like to request. Can you add explicit removal of the "starting" value's key during the "finalize" step here: https://gitlab.com/meltano/sdk/-/blob/1378ed3ede701d2edde1d96fbd1db09d2971e3c4/singer_sdk/helpers/_state.py#L242
Since this is one of those tokens that's only relevant while the stream is in-flight, we should explicitly remove it at the end of the sync operation. (Which is the function of
finalize_state_progress_markers().)For unsorted streams, I think it may automatically show up under "progress_markers" which gets purged as a whole object during finalization. However, for "normal" sorted streams, we won't have a "progress_markers" dict and we should remove the start value if present.