redpanda
redpanda copied to clipboard
Published
20 hours ago •
redpanda-data
redpanda-data
tests: update ManyPartitionsTest
trafficstars
Cover letter
Updates to ManyPartitionsTest:
- Make it dynamically select size based on enviroment: this is useful for developers but also for running on different instance types without having to manually adjust things.
- Configure retention rules so that the test can run as long as it likes without filling disk.
- Add a compacted topic test
- Add a test that runs an OMB workload against a system with many partitions
- Use new kgo-repeater traffic generator
Other improvements:
- The KgoRepeaterService comes with a sweet python context manager for running part of a test with background traffic.
- Check for XFS on dedicated nodes: this saves developer time if they've accidentally broken a /var/lib/redpanda symlink to ephemeral storage
- Fix for NodeCrash printing
- Many fixes/improvements to test helpers.
Fixes https://github.com/redpanda-data/redpanda/issues/5389
Backport Required
- [x] not a bug fix
- [ ] papercut/not impactful enough to backport
- [ ] v22.2.x
- [ ] v22.1.x
- [ ] v21.11.x
UX changes
None
Release notes
- none
CI failures:
- https://github.com/redpanda-data/redpanda/issues/5378
- https://github.com/redpanda-data/redpanda/issues/5836
- https://github.com/redpanda-data/redpanda/issues/5837
I made it through it! Looks like a few nice scale test is shaping up.
I had a few miscellaneous questions and suggestions, nothing major.
Thanks for making it through a lengthy set of commits.
All the comments I've silently marked resolved are addressed in https://github.com/redpanda-data/redpanda/pull/5970
CI failures were a transient issue apparently https://redpandadata.slack.com/archives/C02LZGSS66M/p1659967842254759