rust-synapse-compress-state
rust-synapse-compress-state copied to clipboard
Meaning of panic messages "Missing {}" in `collapse_state_maps` is unclear
Can you please add more debug info to message "Missing {}"? E.g. which room is related. How to fix this issue? If affected room no longer exists, state events can be removed, right?
which room is related
The compressor only seems to do one specified room at once, so I would have thought it'd be the room you specified.
(In a future where we might process multiple rooms: perhaps the code could be modernised to not panic, instead using anyhow
and the .with_context
function to add the room ID as context to any potential error?)
How to fix this issue?
That's a more of a question. Missing {}
appears in two places, seemingly when it can't find a state group that it expects to find.
I don't fully understand the logic here though.
I would suspect it's either a bug in the state compressor or corruption (missing rows?) in your database.
If affected room no longer exists, state events can be removed, right?
I don't think that this error necessarily means the room is missing.
Forgot to mention, I'm using synapse_auto_compressor, there is no room specified. Therefore I don't know what's the origin of missing state group. If the database is corrupted, I'm looking for a solution how to fix it.
thread 'unnamed' panicked at 'Missing 90660': src/lib.rs:651
Traceback (most recent call last):
File "/root/matrix/./auto-compress", line 5, in <module>
synapse_auto_compressor.compress_state_events_table(
pyo3_runtime.PanicException: Missing 90660
Oh I see, I wasn't aware of synapse_auto_compressor; that's a new addition since I last used this repo. Having that information seems like it would be useful so looks like it should be added.
In the meantime, you could find out what room is for a state group using SQL:
SELECT room_id FROM state_groups WHERE id = 90660;
As for figuring out whether it's a bug in the compressor or database corruption on your end, it would need investigation.
Unfortunatally it doesn't find any room with that state group :(
Got a similar issue:
[2022-01-07T09:21:41Z INFO synapse_auto_compressor] synapse_auto_compressor started
[2022-01-07T09:21:41Z INFO synapse_auto_compressor::manager] Running compressor on room !qZFigcWoZoRODhzUlw:matrix.org with chunk size 500
[2022-01-07T09:21:41Z ERROR panic] thread 'main' panicked at 'Missing 5817': src/lib.rs:666
I've got this issue too:
[2022-06-16T02:02:00Z INFO synapse_auto_compressor] synapse_auto_compressor started
[2022-06-16T02:02:00Z INFO synapse_auto_compressor::manager] Running compressor on room !sgvHsecpjxfyPEWmiK:fosdem.org with chunk size 500
[2022-06-16T02:02:00Z ERROR panic] thread 'main' panicked at 'Missing 376285': src/lib.rs:666
Same here:
synapse_auto_compressor -p postgresql:// -c 500 -n 200
[2022-07-24T07:43:11Z INFO synapse_auto_compressor] synapse_auto_compressor started
[2022-07-24T07:43:11Z INFO synapse_auto_compressor::manager] Running compressor on room !GtIfdsfQtQIgbQSxwJ:archlinux.org with chunk size 500
[2022-07-24T07:43:16Z ERROR panic] thread 'main' panicked at 'Missing 37272': src/lib.rs:665
Rather than having a descriptive error message, i would prefer it if synapse_auto_compressor
still continues with the other rooms.
Unfortunatally it doesn't find any room with that state group :(
You can run it with RUST_LOG=debug ./synapse_auto_compressor ...
Which gives you some error like:
[2022-07-27T11:32:18Z DEBUG tokio_postgres::prepare] preparing query s29:
SELECT target.prev_state_group, source.prev_state_group, state.type, state.state_key, state.event_id
FROM state_group_edges AS target
LEFT JOIN state_group_edges AS source ON (target.prev_state_group = source.state_group)
LEFT JOIN state_groups_state AS state ON (target.prev_state_group = state.state_group)
WHERE target.prev_state_group = ANY($1)
[2022-07-27T11:32:18Z DEBUG tokio_postgres::query] executing statement s29 with parameters: [[75275]]
[2022-07-27T11:32:18Z ERROR panic] thread 'main' panicked at 'Missing 77986': src/lib.rs:665
The 75275
here is the id for
select room_id from state_groups where id = 75275;
and you can then delete compressor_states for it with
delete from state_compressor_state where room_id = '[roomid]';
delete from state_compressor_progress where room_id = '[roomid]';
which should make it work again
I got the following:
[2022-10-24T20:32:52Z DEBUG tokio_postgres::prepare] preparing query s11:
SELECT m.id, prev_state_group, type, state_key, s.event_id
FROM state_groups AS m
LEFT JOIN state_groups_state AS s ON (m.id = s.state_group)
LEFT JOIN state_group_edges AS e ON (m.id = e.state_group)
WHERE m.room_id = $1 AND m.id <= $2
AND m.id > $3
[2022-10-24T20:32:52Z DEBUG tokio_postgres::query] executing statement s11 with parameters: ["!dlPhxKlwObeRkhUZCm:maltee.de", 969, 963]
[2022-10-24T20:32:52Z DEBUG synapse_compress_state::database] Got initial state from database. Checking for any missing state groups...
[2022-10-24T20:21:34Z DEBUG tokio_postgres::prepare] preparing query s12:
SELECT target.prev_state_group, source.prev_state_group, state.type, state.state_key, state.event_id
FROM state_group_edges AS target
LEFT JOIN state_group_edges AS source ON (target.prev_state_group = source.state_group)
LEFT JOIN state_groups_state AS state ON (target.prev_state_group = state.state_group)
WHERE target.prev_state_group = ANY($1)
[2022-10-24T20:21:34Z DEBUG tokio_postgres::query] executing statement s12 with parameters: [[968]]
[2022-10-24T20:21:34Z ERROR panic] thread 'main' panicked at 'Missing 963': src/lib.rs:673
but
synapse=# select room_id from state_groups where id = 968;
room_id
---------
(0 rows)
How should I proceed?
edit: I deleted the room !dlPhxKlwObeRkhUZCm:maltee.de
from the statement before, that worked.