redpanda
redpanda copied to clipboard
Feature: offline_log_viewer updates
Cover letter
These changes add a first support in tools/offlile_log_viewer for serde-encoded controller messages.
running
python viewer.py --type controller --path [redpanda data path]
will decode logs produced by v22.2 and earlier. serde commands start with a field 'envelope' that contains
[version, compat_version, encoded_size].
Partially fixes #5293,
implements:
- [x] serde decode of controller commands:
- [x] decode_topic_command
- [x] decode_user_command
- [x] decode_acl_command
- [x] decode_config_command
- [x] decode_feature_command
- [x] decode_node_management
- [ ] gold master and coverage testing
- [ ] ductape integration
- [ ] test that new version of redpanda can be completely decoded with this tool
- [ ] add means to run this tool as a part of a ductape test
open questions:
- should there be a check (with a message) that all the bytes are processed?
- should the structure of the json reflect more closely the cpp class hierarchies (some hierarchies are flattened)?
- should the names of the fields match the one used in the cpp source (some names are abbreviated or de-abbreviated)?
Backport Required
- [ ] not a bug fix
- [ ] issue does not exist in previous branches
- [x] papercut/not impactful enough to backport
- [ ] v22.2.x
- [ ] v22.1.x
- [ ] v21.11.x
UX changes
messages are decoded in a stream of json files. the schema for adl encoded files is unchanged, but the schema for serde - encoded message is different in some places than the corresponding adl-one, reflecting more the hierarchical data organization.
Release notes
- none
@andijcr can you merge https://github.com/redpanda-data/redpanda/pull/5315 into this branch as a starting point for tests? I just rebased that PR but I expect it to fail without the changes in this PR to add serde support.
(didn't review the code but I notice the commit messages aren't in the usual format: see CONTRIBUTING.md in the source and/or take a peek at the commit history)
@andijcr this has been stalled for a while -- it's not our top priority, but it would be good to know if there were issues blocking this
I've merged this into https://github.com/redpanda-data/redpanda/pull/5315