questdb icon indicating copy to clipboard operation
questdb copied to clipboard

chore(core): reduce sequencer txn log file storage record length from 60 to 40 bytes

Open ideoma opened this issue 11 months ago • 1 comments

The new sequencer V2 record can be compacted to 40 bytes instead of 60 without sacrificing too much of usable data range and code complexity.

Proposed layout:

  • Structure Version: 40 bits, range: 0-1T
  • WAL Id: 32 bits, range: 0-4B (unchanged)
  • Segment Txn: 24 bits, range: 0-16M
  • Segment Id: 24 bits, range: 0-16M
  • Commit Timestamp: 48 bits, stored in milliseconds, range: years 1970-10889
  • Min Data Timestamp: 64 bits, microseconds
  • Max Data Timestamp: 48 bits, stored as the difference from Min Data Timestamp, microseconds. Range: 3 years from Min Data Timestamp
  • Commit Row Count: 40 bits, range: 0-1T

ideoma avatar Mar 21 '24 11:03 ideoma