nips
nips copied to clipboard
nostr-protocol
NIP-65 (Relay List Metadata) should only contain write relays, not read relays
Right now there's no scalable way to find out where an author writes its events.
NIP-65 tries to fix it by proposing an event kind that contains all reader and writer events. It tries to achieve two goals in 1 kind: relay readers and writers, even though their access patterns are extremely different.
Reader relays are interesting for clients to know where to read events from by default, and for other relays to try to optimize where they write their events to. This information is very rarely accessed.
At the same time write relays are important for finding events that an author replied to. Also there are usually significantly less write relays than read relays.
I propose to split this information into 2 kinds, and just have the content a JSON array of relays instead of using tags to save bandwidth.
This arrangement makes it easy to have write relay list metadata servers, that scan all relays for write relay events, store all of them in RAM, and serve only those events, thereby acting similarly to domain name servers on the internet.
The great thing in this solution is that the relays can be stored in a list by the servers, and I believe no more than 100 bytes are needed for storing all write relays for an author in this compressed form (signature data and author pubkey are the most expensive parts). This allows storing write relays for 1 billion authors on a server with 100GB RAM (which can be improved by sharding in the future).
The most expensive part of serving the data is bandwidth (RAM hash lookups are practically free), that's why I propose this simple form for write relays (tags are not allowed):
{
"kind": 10002,
"content": "[\"wss://alicerelay.example.com\",\"wss://brando-relay.com\",\"wss://expensive-relay.example2.com\"]"
...
}
Read relays could be kept in kind 3 like they are being done now, or we could store them in a different kind in the store format (I think it doesn't really matter)
One more important thing to keep in mind is that clients SHOULD publish metadata, contact list, and at least fresh data that users probably need to access to new write relays added to this list, so that readers don't have to search events in all these relays.
Either I don't quite understand several things you said, or I disagree, and I'm not sure which. I'm going to write this as if I disagree, but perhaps I don't understand what you meant.
NIP-65 tries to fix it by proposing an event kind that contains all reader and writer events
Perhaps you mean relays? the new event kind doesn't contain events.
It tries to achieve two goals in 1 kind: relay readers and writers, even though their access patterns are extremely different.
It tries to achieve the goal of advertising which relays a person uses. And both read and write are needed information in order to reliably follow and respond to a person. Since they are both needed, it made sense to put them together. In principle, one could separate them, spread their write relays far and wide, and then spread their read relays just to their write relays, but then we'd have a 2-step dance: get their write relay information from just about anywhere, then get their read relay information from their write relays. But there would have to be a good reason to do it this way, and your reason doesn't make sense to me. I can have 15 write relays each with very long URLs. Can you store that in 100 bytes? No. I don't think nostr design should stem from a particular memory arrangement of a particular instance of a particular service being imagined.
But I do think servers that serve kind 10002 events only and specialize as such (as long as there are many of them) is a useful idea.
I think the main misunderstanding we have is in the main goal, all other things are less important. If you think it should be done by a different NIP, I'm happy to propose another.
I'd like a way to know which events to read from where. For example if a client reads the last 200 notes of an author, it contains about 100 replies from different authors. Iris for example shows all events that are replied to in the UI, so it needs to know where to read all those different notes from.
Because of this problem I don't see any practical way of getting rid of the 2-step dance, just like when you browse the web and click on an URL, there's a DNS resolution first that gives back the IP address of the host that can serve the requested resource (which is in itself a 2-step dance).
I don't see any reason for a client to know what relays an author is generally reading from, as it's not displayed in the Iris UI.
We can speak about the storage separately, but I'm more interested in how you would get rid of the 2-step dance problem for this specific problem, if you think it's unnecesary. We should first agree on that.
Ok you are talking about replies to threads. I didn't get that. The 2-step I was talking about was learning about a person's relays which IMHO shouldn't be done over-and-over again as you render a thread, but done periodically ahead of time. But maybe that's too opinionated.
Yes, it is a real problem. I can see Cameri's posts but I can't see what he replied to... gossip doesn't know where to get the event and the 'e' tag didn't have a recommended_relay_url to help it out, and apparently Iris isn't interested in helping other clients (see https://github.com/irislib/iris-messenger/pull/214) climb threads.
But looking up where a person posts is not going to help 'e' tags which specify events, not people. There's no way to accurately coorelate 'p' tags on an event with 'e' tags next to them. I suppose you could look at all the 'p' tags and check all those relays but it seems like a crap shoot and perhaps lots of lookups before you found the event.
I don't see any reason for a client to know what relays an author is generally reading from, as it's not displayed in the Iris UI.
If I want to tag someone in on an event, and they don't follow my posts, I need to place it somewhere that they will see it. I need their 'read' relays for that.
Sure, I understand that, that's why I suggested to have 2 new event kinds introduced.
#e is of course very important, and I agree you, but in my experience not seeing what a person replies to is very frustrating (that's why I'm not using Damus to tell you the truth).
Actually for #e I would maybe place the events to both my write relays and the write relays of the person who's event I'm replying to. This way both clients who are looking for event reactions and clients that are interested in my history can access it easily.
My main point is that a client usually reads 1000x more events than it writes, that's why it's so important to save bandwidth when reading. People have been complaining of clients reading gigabytes in 1 day (which is just not good with mobile data plans). Of course there are improvements already, but splitting to 2 kinds has the advantages without having too much disadvantages.
I see the argument that you don't need to know someone's read relays nearly as often as you need to know their write relays. In fact NIP-65 started out without read relays because I hadn't thought it through entirely when I first wrote it.
But I'm not convinced splitting them would make any significant difference.
I hear you about clients reading gigabytes of data, and trying to limit that, but I don't see how kind-10002 has anything to do with that. Kind-10002 is not something you should be reading gigabytes of. They are short events explaining which relays a person uses. You can load them infrequently and store them in the client somewhere, you don't have to load them over and over.
#e is of course very important, and I agree you, but in my experience not seeing what a person replies to is very frustrating (that's why I'm not using Damus to tell you the truth).
So we both agree it's a real problem. Iris doesn't solve it. It just connects to lots (10 popular) of relays and gets lucky, which wont scale (correct me if I'm wrong @mmalmi )
Actually for #e I would maybe place the events to both my write relays and the write relays of the person who's event I'm replying to. This way both clients who are looking for event reactions and clients that are interested in my history can access it easily.
So are you just coping the entire thread over?