tap-mongodb icon indicating copy to clipboard operation
tap-mongodb copied to clipboard

Multiple ID Types Support

Open dmosorast opened this issue 5 years ago • 0 comments

Description of change

The tap currently cannot support multiple data types in the _id field for FULL_TABLE replication, due to the behavior of MongoDB when comparing differing types (e.g., ObjectId and String). In this case it returns false for both less than and greater than, effectively filtering out the _ids of the differing type from the max _id during full table sync.

This PR contains a broken test that should be fixed by the solution to this issue, in a way that is generic for all users of mongodb and tap-mongodb.

QA steps

  • [ ] automated tests passing
  • [ ] manual qa steps passing (list below)

Risks

Relatively high, this code is hit by a lot of users of the tap.

Rollback steps

  • revert this branch

dmosorast avatar Jun 17 '20 14:06 dmosorast