element-hq
Limited message history unless I increase federated room backfill limit
Description:
I tried creating a new server and setting up federation between the old and the new servers. When a user is invited to a federated room, they don’t see the full room history — it seems that federation only provides a limited amount of history (about 100 messages). The room’s history visibility is set to “since the point in time this option was selected”, so I’m not referring to rooms where history visibility is set to “since they were invited” or “since they joined.”
After checking the Synapse source code, specifically synapse/handlers/federation.py, I found that the function try_backfill has a hardcoded limit=100.
https://github.com/element-hq/synapse/blob/dc7f01f334aa8aa92ca10cdf2e8de64a830effa4/synapse/handlers/federation.py#L446-L540
Potential solutions
Would it be possible to change this limit through homeserver.yaml? Or is there a specific reason why this value is hardcoded?
@s0div The root of the problem likely isn't the limit=100 on backfill.
To explain what's happening, when the client uses the /messages endpoint, Synapse checks if it's approaching a gap in the history, and if so, will backfill more events from the federation to fill in the response. This means your client can paginate the history all the way up to the beginning of the room as it will backfill over and over for each /messages request as you go back in time. Synapse-wise, this works fine for new room joins regardless of the limit used in /messages or backfill, see https://github.com/matrix-org/complement/pull/816 for concrete Complement tests for this scenario.
But there is a similar scenario where you re-join a room that you were previously joined to that does have problems. In this case, if you make a /messages?limit=500 request (where the limit query parameter is larger than the backfill limit=100 we use within Synapse), Synapse will backfill the limit=100 and return those 100 new events + all of the old history leaving an invisible gap in between. So your client goes from 100 new events, [gap], the old history from when you were previously joined. This is what MSC3871: Gappy timelines is trying to address. This is a hole in the spec as there is no way for a homeserver indicate gaps to the client so they can paginate the gap and cause the homeserver to backfill more.
In terms of why you're missing history. Which join scenario are we dealing with (compare to what I explained above)? What client are you using? (Element Web has bugs around this for example)
No, I am not rejoining the same room, it is a fresh room join. I also tried to join the same room with another user, and I am still not seeing the full message history and experiencing the same issue.
As I understand from your answer, by default synapse tries to sync the full message history during the initial sync when a user first joins a room, but this does not seem to work in my case.
The room I am trying to join is not newly created—it has a history of messages. Also, the room has a retention policy and is encrypted, if that helps.
I am having this issue on both the web and desktop apps.
I am having this issue on both the web and desktop apps.
I assume Element Web and Desktop? Both are using the same codebase (desktop is just a wrapped version of the web app).
Element Web has a multitude timeline bugs that could be causing this issue.
@s0div To narrow down a client vs homeserver issue, we would need to inspect the network requests happening. Specifically the /messages requests for the room in question. What would be obvious is if you scrolled up to the gap and no /messages requests were made (client issue).
When it happens, you could Submit debug logs in Element using the /rageshake command (include a link to this issue). I'm not sure if the logs will have enough detail or not but I'll assume the Element Web team can diagnose if they look at them. Otherwise, it will be on you to narrow things down further.
It happened in every room with every user. If it helps, I tried increasing the backfill limit in source code in my testing environment, and it loaded the full room history. If I change it back, some parts of the history have disappeared.
@s0div Please share a set of Synapse logs that include all of the /messages requests before your change and scrollback until you get stuck. And then a fresh set of logs after you make your change and scrollback into the history.
These kind of log lines should be included:
2025-09-30 19:30:47,899 - synapse.access.http.8817 - 521 - INFO - GET-121 - 127.0.0.1 - 8817 - {@anon-20250930_193014-38:127.0.0.1:8839} Processed request: 0.039sec/0.000sec (0.011sec, 0.002sec) (0.005sec/0.019sec/20) 3221B 200 "GET /_matrix/client/v3/rooms/!TGmGiCHpooInlGVpAZ:127.0.0.1:8839/messages?from=s123_10_0_1_1_1_1_85_0_1_2&dir=b&access_token=<redacted> HTTP/1.1" "Perl + Net::Async::HTTP/0.50" [7 dbevts]
Which room ID did you reproduce the problem in?
# `awk '!seen[$0]++'` is giving the unique list (`uniq` only works on adjacent lines being the same)
$ cat ~/Downloads/log-2.txt | grep /messages | sed -nE 's/.*\/([^\/]*?)\/messages.*/\1/p' | awk '!seen[$0]++'
!hAplNFVtMiCazgzwpW%3Aoldserver
!QhXfDRqUnLYdRfAJUo%3Aoldserver
!LuOVJTAkgcZFogwzwK%3Aoldserver
!rFtNXaBoDQcHRLBmVG%3Aoldserver
!BdvJOjGkVmOCDlDFen%3Aoldserver
!hNTCeedoanJCYBQzhC%3Aoldserver
!EFbDTocSuhzsTFoeBG%3Aoldserver
!UPAfBdynNkXhMrkqWO%3Aoldserver
!TgNrOYnDhMllIYHmdn%3Aoldserver
!smuBOrHVhNylzLqHpp%3Aoldserver
!BjqgssxqxXDzXEPYFZ%3Aoldserver
!jkFNdYJRHXAXoTYltG%3Anewserver.com
!ldETomPcsNaCkTzkzJ%3Aoldserver
!TIXfAvTNRYVBFJoMbC%3Aoldserver
!IvpARcSjaficWRTwQC%3Aoldserver
There is a bunch logs with EventTimelineSet:canContain event encountered which cannot be added to any timeline roomId=!ldETomPcsNaCkTzkzJ:oldserver and Decrypted event $xxx is not in room !ldETomPcsNaCkTzkzJ:oldserver: ignoring after the /messages requests for that room. This definitely feels like a client problem. Please create an issue there.
As I suspected, the Element Web client logs are not sufficient to determine exactly what's going on at-least in terms of paginating the /messages endpoint. I've created https://github.com/element-hq/synapse/pull/19226 to add some logs on the Synapse homeserver side in order to better determine whether request pagination is a client or homeserver issue. Element Web should also add some similar logs so people can determine the same thing without needing access to homeserver logs.