lemmy
lemmy copied to clipboard
LemmyNet
Federate lightweight list of communities to ease discovery
For front end issues, use lemmy-ui
Is your proposal related to a problem?
Finding communities is hard. If the instance you're on doesn't know about community X, trying to search for it will result in nothing. If a community is new to the instance you're using, you have to search twice for the community to find it.
Describe the solution you'd like
Instances can request and share a list of communities that they know about.
This could be done infrequently to reduce network load.
This could be opt-out on an instance-level as well as a per-community level. Maybe a simple checkbox
[ ] Enable global discovery
Describe alternatives you've considered
(Write your answer here.)
Additional context
The biggest issue (other than scaling, which it sounds like some helpful hands are working on) with user adoption almost certainly boils down to a short list of items:
- Lack of client knowledge about how federated networks work (out of scope for this issue)
- To Lemmy's credit, it's made a bit more clear here than on other networks, which is great - this is out-of-scope for this issue anyway.
- Slightly cludgy mobile UI (out of scope for this repo)
- It would be an absolute godsend if Apollo and Relay both created lemmy-centric forks of their apps. Anyway...
- Possibly most important: lack of discovery.
What do I mean by "Lack of discovery", you may ask? Simple: Finding and subscribing to communities outside your current instance is extremely user-unfriendly and cludgy. This is caused by a myriad of different "papercuts", none of which are particularly bad on their own but together make it very difficult for even a power user to discover and subscribe to communities external to their own instance.
- Links to communities and posts almost never link to a locally-federated copy of the contents unless special care has been taken to do so.
- The backend doesn't auto-fetch the necessary data AFAIK, so if it's not already federated it will result in an error page.
- There's no easy in-instance way to discover communities that somebody hasn't already subscribed to on the instance. Not great for small or single-user instances.
- There's no quick way to subscribe to a community that is linked to.
- This is more a UI problem I think, but adding a small inline subscribe button (maybe with just an icon, and completely omit the button if you're already subscribed) after community mentions in post bodies, comment bodies, and sidebar would do wonders. At the very least, it would allow instance owners or community owners to create quick lists of related communities that people could join.
- Fresh users aren't subscribed to anything by default.
- Allowing instance owners to auto-subscribe users to a predefined list of communities would be a good first step
- Expanding that idea, having the user complete an "account initialization" phase where they can select which communities to subscribe to out of that list, and also enable them to search for communities across the "LemmyNet" via category, tags, name, and short description.
- More info on that idea: https://github.com/LemmyNet/lemmy-ui/issues/1106
This friction is, today, encouraging instance "siloing" where users are piling on one instance because the fediverse is confusing to them and Lemmy doesn't make it very easy to find new federated communities unless the instance has seen them before.
All that said, having instances share their list of known communities that want to be found is a great first step to enabling fast-and-wide discovery.
To provide a bit more context:
The issue comes from the first time a user searches for a community from instance B their instance A imports 20 posts from that community.
So the "second search" is usually done after instance A has finished the import, but it might be before depending on how long the import takes (connection speed, server load, etc).
The default message of "No results" seems to happen because the search is done in two parts.
- The UI sends a "Search" operation which seems only retrieves data from the DB.
- The UI sends a "ResolveObject" operation which returns the community if in the correct format as per the documentation. So if the users sits long enough in the search page with the message "No result" it'll eventually disappear after the import was done and the backend sends the response to this operation.
The first improvement could be to queue the import and immediately return to the UI the community.
A more complex improvement I imagine is have a setting on how to handle the import of communities:
- Lazy: Current behavior with the improvement mentioned above.
- Communities names: When a user searches for community X from instance B for the first time in instance A the community and 20 posts are imported, but also all the names of the all the communities from instance B.
Then when another user searches for community Y from instance B the name is returned from the DB of instance A and now backend imports 20 posts of community Y.
This will greatly improve user experience as searching for community Y doesn't require the format, it can be searched just by name.
Also include an "Import now" option: admin provides an instance to fully import, all communities and 20 posts from it.
This will require some rate limit so instances don't overload another instance.
It would be possible to federate a list of all communities for each instance. The problem is that all of them would have to be stored in the local database which might be a lot of data. Also the community list on the website would get way too long.
Most other problems you describe are unrelated though, there are probably existing issues already.
It would be possible to federate a list of all communities for each instance. The problem is that all of them would have to be stored in the local database which might be a lot of data. Also the community list on the website would get way too long.
Most other problems you describe are unrelated though, there are probably existing issues already.
Agreed - most of the other issues I listed are largely unrelated.
Maybe limit the community list federation to just communities that have been used for content (post, comment, creation) in the last 30 days. I can't imagine the DB entries would take much space relative to the rest of the content the network serves, and community icons could be forcibly resized down even further.
On a side note, I appreciate all the work you and others have put in here, it's a really slick product and easily one of the best open source social network-type software I've ever used.
I think the lemmy-ui's could very much benefit from a "global community discovery service" like https://browse.feddit.de , but integrated into the front ends. I'd of course prefer that each lemmy back-end do their own crawling of communities and instances, to make it as distributed as possible.
Other ActivityPub services "solve" this problem by allowing instance admins to subscribe to relays. Works quite well with Akkoma/Pleroma for example.
The Pict-rs developer seems to have developed a AP relay service in Rust that would probably be a natural fit for Lemmy: https://git.asonix.dog/asonix/relay
I suggested something similar to @poVoq a while back.
Basically, an instance would Announce community Create messages to any relays that it follows and the relay would Announce that Create to all of its other followers.
Mastodon also uses relays for discoverability. Bigger instances forward toots to relays, who then forward them to subscribers.
