mastodon
Provide more than 400 posts in home timeline
Pitch
For people who are heavy users of Twitter (following many people) and are used to plowing through long timelines, the 400-post limit on home timelines in Mastodon can come as a surprise (it did to me). If you let the number of posts you haven't looked at get over 400, suddenly you can't ever see the ones you missed again.
Any of the following would be an improvement:
- Keep 24 hours of posts
- Keep a numeric limit, but up the limit to 1000 or more
- Keep the current limit of 400 posts in memory, and fetch older data from disk. This kind of access is rare and it's OK if it's slower.
- Change the official app, and suggest to third party apps, to background fetch new posts and keep them cached in a local file.
Motivation
On Twitter, I am a timeline completionist. I read the whole thing, and I follow a lot of people (there's a lot of skimming). I can go through 1200-1300 tweets in a day (I'm trying to cut back now that Mastodon is busy).
Mastodon currently limits your home timeline to the most recent 400 posts. Normally this is not a problem if you use an app, because the app keeps much more than that. However, all the apps (currently) discard all their data on a restart. I found this out the hard way when I rebooted my device and I had a 5-hour hole in my timeline there was no way to catch up on.
I should add I've never, ever had this problem with Twitter using apps like Tweetbot, which allow unlimited access to the timeline. Other apps like Twitteriffic have a limit, but it's 1000 tweets. People coming from Twitter may be surprised, as I was.
I like this idea although I suspect there is probably some significant amount of server overhead with allowing users to retrieve a very large (or indefinite) amount of timeline at will.
Another approach that might help with this problem for "timeline completionists" is https://github.com/mastodon/mastodon/issues/18601 as it would allow you to have a static subscription to your timeline and fetch it into your own RSS reader for permanent storage.
I'm curious, is keeping the feed hydrated in Redis that much more performant than hydrating it in a similarly denormalised (same JSON blobs) table?
I can definitely understand why you'd want to significantly limit the entries in Redis (memory is expensive) but DB storage is significantly cheaper and a feed is only being accessed by that user.
Only by changing that constant in the code.
Thank you for your swift answer! Could you perhaps point me to the correct file? And also, would it be possible to simply shut down the server briefly, change the constant to another value and start it back up? Or would it require some more work?
You can do this by editing app/lib/feed_manager.rb and increasing MAX_ITEMS: https://github.com/mastodon/mastodon/blob/f77ed3a6b202b0eba42e4ce2e76b09dcf70398b4/app/lib/feed_manager.rb#L10
Maybe close this in favor of #8336 ? ~~Infinite timeline is already working in Glitch Fork: https://github.com/glitch-soc/mastodon/pull/1610~~ Apparently Glitch Fork doesn't have this feature, it was reverted.
Why is there even a limit? Isn't rate limiting enough, the SQL query can't be that expensive.
IMO Rate limiting is the right way, if it's expensive let's make you can do ~10 queries per minute, not hard limits on counts.
The current hard limit makes it impossible to catch up on the home timeline if you have been away for a day.
Now I'm coming up with all kinds of hacky schemes to overcome this:
- Iterate all users and fake home timeline
- Add all users to lists of ~30 (partitioning them)
If you follow just a few prolific people, that 400 posts limit comes very soon. Drowning out other people.
