Aaron Fabbri

Something similar [here](https://buildkite.com/redpanda/redpanda/builds/14356#0182b4b0-8675-423e-8927-fb4d5733d227). I expected to see the newer named semaphore output, then realized the error may be from the pre-upgrade code.

Seen in CI failures today [here](https://buildkite.com/redpanda/redpanda/builds/13205#01824129-9475-4e1d-a674-20f440976b7b).

Showed up in CI failures [here](https://buildkite.com/redpanda/redpanda/builds/13206#0182412b-be8f-4001-b2f5-061d81c39390), reopening.

Missing manifest here was `40000000/meta/kafka/topic-unubmkojgx/57_37/manifest.json` ``` RedpandaService-0-140401461113088/ip-172-31-47-80/redpanda.log 8621:INFO 2022-08-09 20:35:07,197 [shard 2] storage - segment.cc:614 - Creating new segment /var/lib/redpanda/data/kafka/topic-unubmkojgx/57_37/0-1-v1.log 9023:DEBUG 2022-08-09 20:35:07,244 [shard 2] cloud_storage - [fiber175~0~0|1|10000ms] - remote.cc:203...

Searching for this particular partition in the logs, it looks like this may just be that the affected partition didn't have any non-data batches? _edit: This is very unlikely, given...

I'll follow John's comment above with another `r/heartbeat - heartbeat_manager.cc:284 - cannot find consensus group:5` error in this test [here](https://buildkite.com/redpanda/redpanda/builds/13165#01823e51-fc1f-44c0-92e5-a8653924cd93).

> There very much is already a cluster quorum: that's the controller raft group. Yep, this is clear. I don't think topics and partition metadata has to be a part...

Adding some code links for reference, for when we get around to discussing this topic. - Bit of [consensus code](https://github.com/redpanda-data/redpanda/blob/dev/src/v/raft/consensus.h#L511-L533) that illustrates that we don't know whether or not we...

IIUC, #5884 is related to the lack of cluster health invariant.

#5949 looks related per this [comment](https://github.com/redpanda-data/redpanda/issues/5949#issuecomment-1212121628).