strfry icon indicating copy to clipboard operation
strfry copied to clipboard

NIP-114: ids_only filter

Open mmalmi opened this issue 1 year ago • 8 comments

When a subscription has a filter with ids_only: true, the relay responds with [HAVE, subId, eventID] messages. The client can then request the full event from only one relay instead of receiving it from all subscribed relays. If the event is already in a local database, the full event request can be skipped altogether.

I have not implemented this for the stream command yet, but this could save a lot of bandwidth if you're connecting to many other relays and subscribing to everything.

mmalmi avatar Jan 31 '24 10:01 mmalmi

We might want a GET message that is sent in response to HAVE messages to request the full event without creating a subscription.

mmalmi avatar Jan 31 '24 10:01 mmalmi

Added ["GET", eventId] handling. Response is sent as ["EVENT", "*", eventJson]. Is "*" a good subscription id here or should it be something else?

mmalmi avatar Jan 31 '24 12:01 mmalmi

Thanks for this! I'm looking into it. My first impression is that negentropy would be a more optimal solution.

hoytech avatar Jul 19 '24 03:07 hoytech

Negentropy is better for past data, but doesn't work for ongoing subscriptions. Both can be used, but this is a lot easier to implement in all clients.

mmalmi avatar Sep 01 '24 19:09 mmalmi

I think this is a pretty good idea. Did you try starting a discussion on this in the NIPs repo? I'm a bit hesitant to add non-standard protocol extensions unless there is some consensus.

hoytech avatar Sep 05 '24 22:09 hoytech

Found it: https://github.com/nostr-protocol/nips/pull/1027

Yes I agree, this solves an issue that negentropy does not, and is in fact complementary to it. You could do an initial sync with negentropy and then stream new IDs down as they come in, without duplicating the download of full events. In fact, this may be quite useful for a nostr multiplexer I am working on.

Let me think about this a bit more, but I'm leaning towards merging this. Thank you!

hoytech avatar Sep 07 '24 00:09 hoytech

I left some comments in the NIP above. In short, I think the GET/HAVE functionality should be separated from the ids_only: IMO they are better accomplished with existing functionality.

I also think ids_only should somehow be a property of the request, not an individual filter. If some filters in the same REQ have ids_only and others don't, it will be unpredictable which receive the full event versus just the ID.

hoytech avatar Sep 08 '24 14:09 hoytech

Thank you, good points. I can help if needed, but you're more familiar with the code and can probably implement it better.

mmalmi avatar Sep 16 '24 10:09 mmalmi