risingwave
risingwave copied to clipboard
ch-benchmark q9 data verification fails in chaos mesh test
https://buildkite.com/risingwave-test/chaos-mesh/builds/590#018dcd0c-7f3b-4bb6-92ac-719ee95386d2
The experiments made the meta pod fail for 20 seconds
Didn't see the ERROR: cannot advance replication slot to 0/C90F020, minimum is 0/15B79B60
log
Difference between this and Q3's inconsistency (https://github.com/risingwavelabs/risingwave/issues/15057): For Q3, the row number are different, but here, both tables have 25 rows. Not sure whether they are the same root problem.
The test instance was gone, so I can't do further analysis.
memtable spill is enabled: https://grafana.test.risingwave-cloud.xyz/d/liz0yRCZz1/log-search-dashboard?from=1708542494000&orgId=1&to=1708547688000&var-data_source=PE59595AED52CF917&var-namespace=longcmkf-20240221-190330&var-pod=benchmark-risingwave-compute-c-0&var-search=mem_table_spill
I think it is similar to https://github.com/risingwavelabs/risingwave/issues/15057#issuecomment-1961343919. It is already fixed in #15232.