add nip 23: relays list.

[fiatjaf]

For sharing relays lists between clients in a reasonable and extensible manner.

https://github.com/nostr-protocol/nips/blob/26-relays-list/26.md

[Giszmo]

So is this meant to help with sharding later on? This relay for kinds x to y, this relay for nwiki, ... ? But any pubkey could advertise any such list of relays and their limitations? What should clients do with those? They will not be consistent between different pubkeys and who would share them based on what? Based on relays self-reported filters?

I see this nip as benign but don't see the benefit yet.

[cameri]

I see this nip as benign but don't see the benefit yet.

@Giszmo There's currently no NIP specifying how a client should store a list of relays. Branle stores the list of relays in the content field of kind 3 events but that's not covered in NIP-02. This NIP fixes that.

[Giszmo]

I find the rules quite complex and doubt many clients will implement all of it. I agree with @Cameri to not use #. (I don't agree with dropping ! and adding ^.)

In general it's an ACK from me. Clients that are confused will figure out what to do with the list of relays either way.

[cameri]

^ is already part of the runes-like spec and means starts with or begins with. It can be skipped for now since adding support for it won't break existing rules. I think it is fine to implement a subset, just what we need. But our usage of # and ! are not part of the original runes-like spec and would make it non-compliant for whatever it's worth, breaking compatibility with existing libraries. Anyone wanting to implement this NIP will have no choice but to implement the custom spec, not a hard problem true, but they could have just leveraged the existing python library for example. Either way this not a deal breaker for me but I don't find it ideal. ------- Original Message ------- On Sunday, August 14th, 2022 at 12:07 AM, Leo Wandersleb @.***> wrote:

I find the rules quite complex and doubt many clients will implement all of it. I agree with @.***(https://github.com/Cameri) to not use #. (I don't agree with dropping ! and adding ^.)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[KotlinGeekDev]

Hello everyone. This spec is interesting(but complex, at least to me). Though I have partially understood, thanks to the examples used, and the event structure, I fail to understand the use of evaluation criteria in the examples(where conditions such as pubkey=(a value), etc are determined). How are they used? Let us imagine a scenario between 2 Nostr users. One person shares the list. Does the other person get the list with the values defined, or with the evaluation criteria? Thanks.

[jb55]

The first use case is meant to make it so users can open their client -- or different clients -- in different devices and have their list of relays automatically fetched from a default relay and start using their relays list without having to set everything up again.

isn't this is what the contents of kind3 is for?

[eskema]

The first use case is meant to make it so users can open their client -- or different clients -- in different devices and have their list of relays automatically fetched from a default relay and start using their relays list without having to set everything up again.

isn't this is what the contents of kind3 is for?

well, that was a rogue undocumented implementation in Branle, we kinda discussed this in the chat group that a new NIP dedicated for relays would be better.

[fiatjaf]

Branle is now implementing a draft of this proposal that only recognizes empty strings as true.

[jb55]

On Sat, Aug 13, 2022 at 08:20:18PM -0700, Ricardo Arturo Cabral Mejía wrote:

I see this nip as benign but don't see the benefit yet.

@.*** There's currently no NIP specifying how a client should store a list of relays. Branle stores the list of relays in the content field of kind 3 events but that's not covered in NIP-02. This NIP fixes that.

I see the benefit of a spec as a standardization of nip-3 content but I'm not exactly sure what I would do with the rules as a client.

[cameri]

On Sat, Aug 13, 2022 at 08:20:18PM -0700, Ricardo Arturo Cabral Mejía wrote:

I see this nip as benign but don't see the benefit yet.

@.*** There's currently no NIP specifying how a client should store a list of relays. Branle stores the list of relays in the content field of kind 3 events but that's not covered in NIP-02. This NIP fixes that.

I see the benefit of a spec as a standardization of nip-3 content but I'm not exactly sure what I would do with the rules as a client.

Instead of clients blasting events to all relays the write rule lets users choose which relays to send to per event basis. Instead of subscribing to all relays with the same filters, the read rule lets users choose which relays can be subscribed to (the filter must match the read rule).

[cameri]

What's missing here to move forward?

[fiatjaf]

Check if these rules make sense from the client point-of-view or determining what rules make sense.

I think the current form is not great. Maybe we could just specify an arbitrary JSON object of relay URLs with an object for each like

{
  "wss://relay.nosotros.com": {"read": true, "write": true},
  "wss://personal-relay.x.com": {"read": false, "write": true, "archive-everything-i-like": true}
}

and then clients interpret that in no strict manner.

Then we standardize later the fields that end up being used.

[dskvr]

Maybe we could just specify an arbitrary JSON object of relay URLs with an object for each like

stringified JSON in content?

[fiatjaf]

Yes. Very bad?

[dskvr]

Nope, just wanted clarification.

I actually see a benefit to this, unlike tags, there's less chance of error for clients to add a relay twice since it's a dictionary. Granted, anynostr library supporting pools would probably handle this, but it potentially keeps the data from getting out of hand and persisted due to a bug in a client.

Clients won't have to implement logic to guarantee uniques and clean up the data.

Also, as revised, this resolves #117's stated needs and objectives.

[KotlinGeekDev]

Please, @fiatjaf would it not better to make it a JSON array of relays? Something like [ { "url": " wss://... ", "read": true, "write": true}, { "url": " wss://...", ...}, ...] I think it would make it easier to parse for clients.

[fiatjaf]

@KotlinGeekDev that is true, but then you would have to handle duplicates in your code. What do you prefer?

[dskvr]

@KotlinGeekDev

Using JS as an example

const relays = JSON.parse(event.content)
for(const relay of Object.entries(relays)) {
key = relay[0]
policy = relay[1]
...
}

vs

const content = JSON.parse(event.content)
const uniques = new Set()
for(const relay of content.relays) {
if(uniques.has(relay))
  return 
uniques.add(relay)
key = relay.url
policy = relay.policy
...
}

Not much different, except that not every language has easy uniques from Array, whereas every language that supports the concept of dictionaries inherently supports unique keys. For an implementer, understanding that the result may not be only uniques is something that many people will forget to implement, and adds friction to adoption.

Objects have unique keys by default.

Doesn't completely eradicate the problem, however, anomalies such as trailing slashes will still require sanitization. Which brings up another point, the standard should establish whether relayUrls should or should not contain a trailing slash.

~The argument for arrays actually gains some credence with respect to trailing vs no trailing.~ Given the result...

event.content:stingified {
  "wss://myrelay" : { ... },
  "wss://myrelay/" : { ... }
}

We would need to dedupe, with Objects would require something a little messy, such as

const content = JSON.parse(event.content)
const uniques = new Set(Object.keys(content).map( sanitizeFn )}) //return only keys
uniques.forEach(relayUrl => {
  content[relayUrl]
})

Or if the standard is strictly expressed, then the client could reject/remove trailing/no-trailing depending on the standard. While strict would be better for devs, it is bound to bother some users

But if it were an array, ~this particular situation would be easier to mitigate, but still messy (for most scripting languages, at least)~ No it wouldn't, appears more messy. Got mixed up, now you have to filter on each iteration to select the correct item from the array.

Given the data

event.content:stingified {
  relays: [
    {url: "wss://myrelay", policy: { ... } },
    {url: "wss://myrelay/", policy: { ... } },
  ]
}

const content = JSON.parse(event.content)
const uniques = new Set()
content.relays.forEach(relay => {
  relay.url = sanitizeFn(relay.url)
  if(uniques.has(relay.url))
    return 
  uniques.add(relay.url)
  //relay should now be unique
  ...
})

There are benefits on both ends, a schema that can validate the object would probably be good no matter what is decided.

Another point for arrays is that schemas generally don't support validating keys, they expect objects to be well-formed, dynamic keys tend to break schemas, so custom solutions would need to be developed for any target language.

As Object:

• +1 Uniques are inherent
• +1 much simpler implementation
• -1 More difficult to find uniques/sanitize
• -1 Programmatic schema support is more limited

As Array:

• +1 Programmatic schema support
• -1 More difficult to dedupe
• -1 Requires extra steps process each relay

Scores from summary As Object: 0 As Array: -1

O_O

[KotlinGeekDev]

@fiatjaf I already handle duplicates in code when it comes to obtaining notes from several relays. So it is not a big deal.

[KotlinGeekDev]

But @dskvr raises some interesting points here as well.:thinking:

[dskvr]

@KotlinGeekDev Made some logical errors, updated. Unlike an object, it would be difficult to easily/efficiently dedupe an array of objects after sanitization. You would either need to filter the array on each iteration or iterate the array to dedupe. With an object, this is inherent functionality.

[dskvr]

I already handle duplicates in code when it comes to obtaining notes from several relays. So it is not a big deal.

I think we're more concerned what it means at scale, on a case by case basis anything is solvable.

[dskvr]

On second look some of the code examples I gave were grossly incorrect (probably still are, they are untested), but they're much closer correct now. Deduping an array of objects is objectively more cumbersome. Only real benefit to Array of Objects is that objects with predictable keys provide wider support for programmatic schemata (json-schema, protobuf, etc)

[KotlinGeekDev]

Right @dskvr , I see your point. Though, it will be complicated to implement that parsing logic.:sweat_smile:

[dskvr]

@fiatjaf As modified this is basically kind:3 but in the replaceable-kind space. If you like your proposal changes, the similarities simplifies things. Coracle appears to be using kind:3 to track user relays, though, doing so is quite messy as User Relay Lists are really a user state variable [replaceable kind], not a recommendation. I would love to see some traction made here and for this replaceable-kind to see the light of day. Let me know if there's anything I can/cannot do to help.

[fiatjaf]

I am also interested in seeing some traction here. I am not sure how to proceed since I am not implementing this myself right now, so maybe we should again ask for the input of the clients who are currently using these other approaches and what would be best for them.

[staab]

I think the array syntax as proposed is fine, a dictionary with a list of policies would also work. I'm not sure the use cases hold up though, they seem to imply that notes are distributed across relays based on use case, rather than sphere of contacts. I think we're ultimately going to end up with a network topology divided by social graph (like mastodon) rather than by function (microservices).

For that reason, the DM use cases don't seem to make sense to me — if you're limiting the relays you send DMs to without reference to which relays the person you're trying to contact is connected to, you're making deliverability worse. Clients should be smart enough to send DMs only to relays the recipient is known to participate in instead.

For the "Bob's/my personal relay" use cases, I would expect that to be implemented on the relay's end instead. Event publishing is not very high-volume, a personal relay can simply drop anything its not interested in (and it'll have to anyway to reject spam, unauthorized pubkeys, etc).

If a relay is full of spambots, I can't think of why you would want to ignore replies but not top-level notes without an additional filter based on web of trust.

So I'm not convinced by any of the filter use cases, much more useful would be to specify an index generated by relays to help clients make smarter social graph decisions: follow/follower/reaction/reply counts, and intersection of pubkeys/relays. I intend to write a NIP for that, but haven't had an opportunity yet.

[staab]

After a little more thought, I do think this NIP could be useful as a hint to other users' clients on how to interact with the user publishing their list. IOW advertising a specific relay as their onbox/outbox could be useful as an optimization so clients aren't just spraying and praying.

And I do think this is an improvement over hijacking kind 3.

[ghost]

Concept ACK

[staab]

I was just going to implement some smarter relay traversal in Coracle and came across the problem this NIP solves, so I'm looking forward to it being merged. I think we should do some cleanup too, since there are some flaws in how relays are recommended currently (see my comment here):

• Kind 2 should be deprecated
• NIP 2 should be modified to specify that kind 3 should include duplicate tags based on the target pubkey's lates relay list. Or some other fix, but a single relay url seems to be a weak point here.

If the filters part of this NIP is still controversial (and remains un-implemented), I vote we split it out into a separate PR.