paimon
paimon copied to clipboard
[Bug] Sequence group may cause record in wrong state.
Search before asking
- [x] I searched in the issues and found nothing similar.
Paimon version
If my records come disorderly, -D record may have been compact out of table. Then +I (which sequence number is smaller then -D record) may survive.
StreamWriteBuilder builder = getTable().newStreamWriteBuilder();
StreamTableWrite write = builder.newWrite();
write.withIOManager(new IOManagerImpl("~/temp/"));
StreamTableCommit commit = builder.newCommit();
// +I[2025-04-01, 1, type1, 29, 5, name2, 29, 5, name2, null, 9223372036854775802]
GenericRow genericRow = GenericRow.of(
1L,
2L,
2L);
GenericRow genericRow2 = GenericRow.ofKind(RowKind.DELETE,
1L,
null,
Long.MAX_VALUE - 1);
GenericRow genericRow3 = GenericRow.of(
1L,
1L,
1L);
write.write(genericRow);
commit.commit(1L, write.prepareCommit(true, 1L));
write.write(genericRow2);
commit.commit(2L, write.prepareCommit(true, 2L));
readData();
write.write(genericRow3);
commit.commit(3L, write.prepareCommit(true, 3L));
Record should be 1,null,Long.MAX_VALUE-1
Compute Engine
Flink
Minimal reproduce step
above
What doesn't meet your expectations?
--
Anything else?
No response
Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
@leaves12138 In your case, if the sequence number is defined as the seqComparator in the sequence group, the output record should be 1,null,Long.MAX_VALUE-1