meltano
Streams should not be able to add duplicate contexts to state
Migrated from GitLab: https://gitlab.com/meltano/sdk/-/issues/368
Originally created by @edgarrmondragon on 2022-04-27 23:56:26
Summary
There are some situations in which a developer might (accidentally) create/update the stream state with a duplicate partition context. This results in the stream failing on the next invocation when the duplicate is detected: https://gitlab.com/meltano/sdk/-/blob/72e7e2be7271d37c9d2c40ae0d0be3785c243876/singer_sdk/helpers/_state.py#L83-87.
Steps to reproduce
NA
What is the current bug behavior?
Duplicate partition contexts are written to a stream state.
What is the expected correct behavior?
Taps should not be allowed to write duplicate contexts.
Relevant logs and/or screenshots
NA
Possible fixes
One safeguard we could add to prevent this is to check for duplicates before they're added to the tap state in https://gitlab.com/meltano/sdk/-/blob/72e7e2be7271d37c9d2c40ae0d0be3785c243876/singer_sdk/helpers/_state.py#L95-101.
This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.
This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.