redpanda
redpanda copied to clipboard
storage: Refactor segment index
This PR is a preparation step for the introduction of segment index compression. It moves some logic closer to the data from segment_index
to the index_state
. It also extractsindex_columns
class out of the index_state
. The index_columns
component contains three vectors (relative_offset_index
, relative_time_index
, position_index
). Previously, the index_state
maintained them alongside all other state and the index_state
. That makes it impossible to evolve the index because the state is entangled with quite complicated business logic (esp. with timestamp handling). The extraction of index_columns
will allow us to evolve this component independently from index_state
and all associated complexity.
The next PR in a series will introduce different implementation of the index_columns
which will use deltaFOR encoding to save memory.
Backports Required
- [ ] none - not a bug fix
- [ ] none - this is a backport
- [ ] none - issue does not exist in previous branches
- [ ] none - papercut/not impactful enough to backport
- [x] v24.2.x
- [x] v24.1.x
- [ ] v23.3.x
Release Notes
- none