nips
nips copied to clipboard
nostr-protocol
NIP-17 (old 24) Sealed Gift-Wrapped Messages for Private DMs and Small Group Chats
This PR creates a new private messaging scheme from the learnings of many past proposals, including NIP-44, NIP-59, NIP-24, NIP-103, NIP-76.
It brings their best ideas together in a slightly different approach that seems to be better for privacy.
Read here: NIP-17
Nice! But very long haha
I had a lot of trouble dumbing this down even for myself...
Fine for a group of 10 people, absurd for a group of 10,000 people
10,000 events from a phone will be hard. But I can actually see it working for a substack-type of service where the author periodically sends a letter to 100K+ people.
Nice! But very long haha
Yes, @vitorpamplona could we remove one of the two examples? DMs seem like a special case of group messages.
Yes, @vitorpamplona could we remove one of the two examples? DMs seem like a special case of group messages.
We can. I have a feeling that you either implement Group messages from the gecko or don't even try doing this. It would be weird to have DMs but not group messages in this model.
I built a test implementation here, source code is here — I had to make some tweaks to the example code to get it working (a decrypt example would also be helpful).
I'm very excited about this implementation, but do let's get it right. @paulmillr's suggestion about CSV seems worth using. I'd also like to hear thoughts from @earonesty, @jb55 and @fiatjaf.
@staab can you generate a few test cases for us?
Here are two from Amethyst's current code
Receiver's Private Key: de6152a85a0dea3b09a08a6f8139a314d498a7b52f7e5c28858b64270abd4c70
{
"content":"{\"ciphertext\":\"AaTN5Mt7AOeMosjHeLfai89kmvW/qJ7W2VMttAwuh6hwRGV+ylJhpDbdVRhVmkCotbDjBgS6xioLrSDcdSngFOiVMHS5dTAP0MkQv09aZlBh/NgdmyfHHd24YlHPkDuF5Yb4Vmz7kq/vmjsNZvDrTen3TG2DcEoTV9GKexdMEqyBA4LsB2DLnWfpvOi0olDkGjPGSteTaU1nCdOtN8knoEKumrxwevvbygKphorvKX/j3ojMMb0AceJM6Cr6TRIvSsQnKGEv5V8qbC/uIrQoH3N108Fd/2SY2MWuyLKRnuak9F/w82MV13elq8ngyjcktLYM5yrPg5nrxZlyJsV8D7V/g/bvhoL+UmWe0XoCR5LXzy77SfIkgA1ePKEfGp5sD2CVIzXt9zHdFwGxAKZuyB4qwrRaAFrS2xx+Bw4nnEmF6V9NhfheSCmGzTILuTePx4ubvnYw/j8Hmqd6UvM3DBNnlJ3D6po0blirfWvMe/ea+Em4CMXfq8Iq+7r4gRx8azADygKeJ+C89GTBEvS9EvgrXCVfTMVTcFc44YAZhekOqYY1BOZgfxIV4gUiJfpMMd4B9MQv/tmnewrpTsq1reSQQcEW/mXT2cnMeCZbAIJSPg8usZ30QlrH+np+YSzFKWYDP1kThcV0ElEE2Ne8KaUUFIRE5KmhBQc/qtORefCpne5s7V7J5vLjT5rinsDzzENB1XVlmY1Icx42raP5tGAL1gOK5gRHLvtcgFQR3WcDRYaNqELiYxx41j9w9lz5e00Ttla255rZkb760KSLaBFBss6wYGiYCabVgtBNpkExpCFPPEd5eAZa5rNK2QrnojYsdxEnlicF6A+zSChLy/TbzxYwyQywDfoF9F8kBakPZkAhsciQViCii2KlieRq4OgJFZGndmnS82hyPqsoJIm22vWr1iqMvSBHo/9cLj/r+lfmGVOdgM62JHckPZjOLS0QWIb9gQiT+zXZG22+eZElMYbGXVpR1dyMaQtde8ivEVVLas6kMCVKaDTHEFglaCBXjJ3RNJv73HsG1kb0rMmOj8ltbBakjHpv7M59amavuu6SReYt\",\"nonce\":\"6anNjUdNwW6MNfoKzRZcz1R09N1h8G4L\",\"v\":1}",
"created_at":1690660515,
"id":"d90739741c2f5a8c1a03aab5dc219c0b708ed6b0566044495731cd0307cf19a5",
"kind":1059,
"pubkey":"a79b7162f8ebb9c9f7aa65a48977ab7f32aa097520bc543e4d625812154ff6af",
"sig":"9b012504e779632a2a1f55562fa9a85f8ae6245cbc149b83d25b2971249053abc77f65cc068e5d025b871d743678265fede70de4eaf5af642e675a5b6210077d",
"tags":[
[
"p",
"c55f0b0cb4dd180dd4395867b28dd4c208b709144a69fb7c31a46c369a5ad1c6"
]
]
}
Receiver's Private Key: 409ff7654141eaa16cd2161fe5bd127aeaef71f270c67587474b78998a8e3533
{
"content":"{\"ciphertext\":\"Zb0ZNYAcDG5y7BiCWgbxY/i7rN7TxPwr3Oaste6em4VcetuenaMu2SyH6OuCCxxmIa7kFennJD8ZCrev0086azsPNutl9I6OCoOfDQb2GoFaLoJAkE/FuW0uEoEJuN72KsKj05HEjOM6nqL2KiW0pxTCNmlGpweMwpXQdm2ItWkybNpq8+b4NJUDee2czBUd9Kr2ELbPISTYzA17z1IzPXGQw8c73NL+QX9I/QZjM/agqX2x5q11SU52xiRyVd9zHf7TMctZI4QEsqDB6xi54D1bAeZlMhVdcpQRpGDfqRz3KXFlhB3Bwdc8GLgY0aLTn6tJs4qrHP3mQkxFYk0mju0afoc0rloMEUHcBVtM18S9OrTPqfmSqFTQsjaT8g+PkmeiLBo1sXsMCS62w0abSZD9OzQtciMz70ZpcWoLjx5f8panjFClvg4tJ8czMURIHM/IFS1uKAUHBArGN8QpCw8MXQBblpyLDiEkFcSX334Zdps0OIw4z328JSdeejyRh4ks+NHDt9FcjC4iicEqfEh8OTkXuKqEAVkRyfAioNQxWQPnXDzMX0Q+BXvKzBA7NaEBDpbV36H/KnrpBBQwokV9/Byb6Seh3g6GSqRAWD3U6Nk2aBMXkD0xY8vnIqMckBeYHxn8BW7k1FdXFC9lE5xCxWZHkmksJ4f0NVaF37O6d8qOe6RK7bfUeF8/SouJEu+eEX1f4KCMboslwkdk8QA8bThGcRGn8GQBMrPKrpZwHYNyyH8jwt9pywigXJejRLDDnDp3FH/3dbZy5CfuNH6KGydf/O5xx1r316so1UPO1mL5LHJUFZVIaMaMMUsgq12gpI0lLEh5NJPpsi9e3ibkzEZGf7FlAJjJQURbQ8xacN7R+w3GWKbJNHiQbUZ2lXo6fwz33t0DrSqEW970yWPHlqxcpd27EI+qqb5IqfklQZ3RObZZBhzDvImaCPG+U7SmgLhPxnilpGjd5lw/ttiqJhPG9mYFMf1eJXSG+Q9VVkGzN7jxXYtx0q0WGjVq98ZGv5RSnF1d9+QVGCd1fiPS3rsaWdYWly8l0y2quYObJ6Mv3Wh3\",\"nonce\":\"/Q2UTTjVZthm/atcCuDjU1e4reF+ZSgZ\",\"v\":1}",
"created_at":1690660515,
"id":"087d9627d63135a5050758a69222e566c86702e930c9905f0b93ccd6bebeca3f",
"kind":1059,
"pubkey":"e59c00796ae2aa9077fc8bcd57fe8d32c0fc363f7c8b93d055c70804ffff3772",
"sig":"807cb641c314ca6910aaeefadcf87d859137520be1039eb40e39832ed59d456fdd800c5f88bba09e1b395ee90c66d5330847bdd010b63be9919bf091adbc2c2a",
"tags":[
[
"p",
"f85f315c06aaf19c2b30a96ca80d9644720655ee8d3ec43b84657a7c98f36a23"
]
]
}
@paulmillr Did you mean to use a SHA256 of the shared secret? Was this on purpose?
export function getConversationKey(privkeyA: string, pubkeyB: string): Uint8Array {
const key = secp256k1.getSharedSecret(privkeyA, '02' + pubkeyB)
return sha256(key.subarray(1, 33))
}
That was the thing I was missing :)
@vitorpamplona checked your test cases, they decrypt on my end. Here's one from me:
Recipient private key: 7dd22cafc512c0bc363a259f6dcda515b13ae3351066d7976fd0bb79cbd0d700
{
"content": "{\"ciphertext\":\"PGCodiacmCB/sClw0C6DQRpP/XIfNAKCVZdKLQpOqytgbFryhs9Z+cPldq2FajXzxs3jQsF7/EVQyWpuV8pXetvFT9tvzjg4Xcm7ZcooLUnAeAo2xZNcJontN4cGubuDqKuXy5n59yXP1fIxfnJxRTRRdCZ2edhsKeNR5NSByUi+StjV10rnfHt8AhZCpiXiZ/giTOsC4wdaeONPgMzMeljaJWLvl6n11VjmXhkx1mXIQt43CNB1hIqO3p89Mbd9p+nlLrOsR+Xs0TB4DCh4XTPbvgf7B7Z+PgOfl3GZfJy9x6TciLcF4E3Ba1zrPe4f79czCIEiJ1yrIKrzzYvv+it35DZQ8fgveFXpyHnNL29hml8PNjyOsFbCHVYLMGw88evI5PijOcpe1TtdoioX8kX5kVEQSKJXuoSjTorvbRPCgGzaa1m0J0uTpzri5VD22a/Jh2CcAnubg6w4JDdUWCogdSV3NqiJllo7ZF7WnZ3apPdRD23MEfphVBJrcLBUNlmwajnY5IvVTKTkZOP50r9dBapvMWXIo6M6zhy/5vVWJz57863pelYCRG4upaXZuNK9sMBtbiphxmFR83i8RML8KN8Q391Cd/xBN7TxJNo5p2YU25VeGZUAmHY8DYlMQDm8Br0nStAXp3T+DzTRL8FTECa8DJV+KTAPoCxqhv3B28Ehr0XAP75CsHoLU00G48cR7h3vQ0CnfKh6KXU6nnDA5OWfpMYpirACCpsnpSD0OaCQ3gkQp3zZNMS3HcOpnPK/IY7R0esbzgAkvNhkyxaIfPDdf+eRUSOA9+2Ji28MwjjY8Dw3SLdUqCOzIDjQeR/T5oNmaQJm3lZ8G0FxxC6ejD4VJX/NI/x+STeB9jWHWmHZvqKzV6JHNh6qmZb6TKSIPOHpafWFoeJFOmiiigf46sju9vRXmVEAx59HXWnvnvCBNJg877yCMulB6xyQuSdVDuotQU4tQZwCKedTHJ6GqjesM98UlJrDtdWQURwwW1qc7N8tS6PukmUVEf0jmbIWVIBmUlkcVuiSs1g1h1kjt8c4MnGTz3CSgpOd1MqxLrl9WwrTqM+YnE+yeZYUjFoewyKZIQ==\",\"nonce\":\"OdCZczJiUGR4bOGIElQ4UUH4dQmG5U/3\",\"v\":1}",
"kind": 1059,
"created_at": 1690772945,
"pubkey": "e01475e87896800b7285eb0daf263c59f811c8fc5bc8daa105d2c98b6d7c4952",
"tags": [
[
"p",
"b08d8857a92b4d6aa580ff55cc3c18c4edf313c83388c34abc118621f74f1a78"
]
],
"id": "d9fc85ece892ce45ffa737b3ddc0f8b752623181d75363b966191f8c03d2debe",
"sig": "1b20416b83f4b5b8eead11e29c185f46b5e76d1960e4505210ddd00f7a6973cc11268f52a8989e3799b774d5f3a55db95bed4d66a1b6e88ab54becec5c771c17"
}
@vitorpamplona I was working through this topic this morning here, I think it would be worth separating encryption/wrapping/key exchange from use cases, since small and large groups will probably work differently from each other, but will use some common components. What would you think about doing that here? That way we can get NIP 44, 59, and DM/Groups all in one PR.
I think it would be worth separating encryption/wrapping/key exchange from use cases
This PR tried to do the opposite: use a very specific use case to move more abstract concepts along. And I do think that's how NIPs should be written. Abstract proposals are cool and all, but they never go forward by themselves. Implementers need a very clear recipe to follow with a clear goal in mind. Once they implemented these abstract concepts, they can re-use them in other NIPs. It's similar to how NIP04 encryption became the "standard" for many other NIPs.
Merging this PR with a Large Group proposal
I am not opposed as long as it's a simple addition.
My first impression is that large groups are a whole different ball game because they will require some form of content moderation (e.g. #580). And that will trigger all types of pushback from the community ("It's censorship!"). It will be a shame if that pushback taints the current proposal, which has no way to be attacked by censors. Group management is hard and I suspect this NIP will be too big of a hassle to implement if we try to do both at the same time.
@paulmillr What's your read on the use of so many random keys on GiftWraps? Can this become a problem? Do we need additional guarantees to make sure we don't open space for attackers?
This PR tried to do the opposite
That's fair, I was mostly just asking about splitting the concepts out described here into different NIPs, but keeping them in a single PR. If you'd prefer not to that's also fine, we can refine the NIPs later, it just seems cleaner to avoid having people refer to this monolithic NIP for general-purpose stuff like the cryptography stuff, which should avoid minor variations across NIPs.
My first impression is that large groups are a whole different ball game
Having thought about it more, I agree. Have you read my summary of DM proposals yet? I think large groups may be doable on top of this NIP by adding some key exchange stuff on top of it to reduce the number of redundant messages that need to be sent out.
The one other thing that might be worth adding here is something similar to NIP 101, which further hides metadata, and is already implemented by 0xchat. It can be an optional addendum or even a separate NIP that this one refers to as a recommendation. I'm a little worried about proliferating standards, since they can easily result in a client having to implement the superset to have a smooth end user experience.
That's fair, I was mostly just asking about splitting the concepts out described here into different NIPs, but keeping them in a single PR.
If people think it's better, we can break it out. But I don't think we need to make a different NIP for each event kind. Frankly, if this get's people blessings, the text we have today will have to be rewritten anyway to put less emphasis on the Why and more on a very simple How-to.
NIP by adding some key exchange stuff on top of it to reduce the number of redundant messages that need to be sent out.
Do you have a more specific proposal? I generally don't like sharing secrets because it only creates weak points for attackers. But maybe there is a good way to use them.
There are many group types as well. If the usage is like the Large Group on WhatsApp, the group is just a marketing channel for the admins. In this case, I think it would be better to make them use a private follow instead of declaring their membership in some other event.
The one other thing that might be worth adding here is something similar to NIP 101, which further hides metadata
From what I understand, we could gift wrap to a different address. We wouldn't need the entire 101 protocol. One could just "Advertise" to a friend (inside the GiftWrap) a new address that the user is "listening to". We could have 10 addresses per user to download gift wraps for.
It could be an add-on to the current PR. But the only thing it would hide is the amount of GiftWraps directly associated with an account. Diminishing returns for the added complexity.
A simple idea for Forward Secrecy. Users could have a second key hidden somewhere safe and instead of sending the backup messages to themselves, they send to that account. If they lose their main key, their messages won't be found. To recover old chats, just connect the two accounts in the same client.
Do you have a more specific proposal?
Yeah, and it's pretty similar to what you're suggesting. The benefit would be to reduce the number of identifiable recipients by one or two orders of magnitude, and along with that timing analysis and probably other things.
So suppose A and B send 1000 messages to each other over the course of a week. You've got 500 messages each in that span of time (maybe a little more with randomized wrapper timestamps), seen on the same relays. Now let's say that every 100 messages A sends a new recipient pubkey to B via some other kind (10101 in nip 101). B can choose to ignore that and messages will still get delivered, but they can also start sending to the new pubkey. So now you have 101 events each being sent to 10 different recipients. This isn't really a ratchet unless the new keys are sent to the most recent key rather than to the real pubkey and the recipient deletes the key when they're done with it. But you could also up the frequency to the point where you have 2 events sent to 1000 different pubkeys which would be quite obscure. The only thing you could prove is that A was contacted once, by someone, and B was contacted once, by someone.
Like I say, sort of a nice-to-have extension to this NIP for most people (although it would make the heuristic of groups being defined by pubkey more difficult), but it could be used to make a much more private flow for clients that want that (especially if they're willing to communicate the keys out of band, since that would provide forward secrecy), and the same mechanism could be used down the road with shared keys for different use cases.
Users could have a second key hidden somewhere safe and instead of sending the backup messages to themselves, they send to that account.
That's a very similar flow, just applied to self instead of recipient, could work very well.
I'm probably getting too imaginative, I just want to make sure we don't pass over any affordances that other developers would consider to be table stakes (NIP 112 and 101 both have implementations and use this mechanism).
This leads to the creation of Chat Backup Manager micro apps.
Always send to the backup key. But when you need to recover, instead of logging in with the backup key, the backup manager client re-signs all inner content in new wraps with a new key. Use that new key to load your messages in the new client.
It does give users more control: send to self for convenience, send to backup for retention in emergencies, don't send and limit retention to the current device.
This proposal may have specific use cases, but I found it challenging to design large group chats using this privacy solution.
-
To address the issue of sending messages to multiple users, you could simply use the shared public key of the group chat as the recipient. The group members could then decrypt the message using the shared private key. When the group members change, the key could be updated to ensure forward secrecy.
-
Regarding the issue of message recovery, based on the current discussion, it appears theoretically feasible to send additional messages to oneself or to backup services. However, this approach does increase the complexity of the client. I still prefer to have a recoverable event chain.
-
The messages will eventually be packaged into gift-wrap, which can be cumbersome for clients. I can't filter messages by kind or tag, so I must retrieve all events and parse them individually to access the data I need.
In fact, I'm wondering as I implemented in nip101. If we protect the sender's public key by converting it to an alias key and change the alias when necessary, could this provide sufficient privacy protection? This approach could easily solve the problems mentioned above.
@water783 are you planning on large communities in 0xchat? I think this NIP is not intended to cover that use case, but I'd love to work with you on building such a thing down the road.
NIP 101 improves privacy, but doesn't go as far as this NIP. Combining the two approaches I think would result in pretty good privacy in a number of dimensions.
@staab Yes, i am planning to implement a large group in 0xchat. I have a draft NIP, which I’ll share later for discussion.
@water783 I am very curious about what your use case is. Do members of a 10,000 people group know each other's identities? Can anybody add a new person in and see all the past and future messages? Is there a moderator to kick spammers out in case somebody adds one or in case an attacker gets a member's key?
@vitorpamplona Yes, That's about right, but those who join the group chat can't see previous messages, and those who leave can't see the future messages. And could consider a rapid key rotation strategy where all the group members get a chance to rotate the shared group key.
Sorry, to be clear. You mean:
- Members of large groups do know each other's identities.
- There is a moderator to kick people out and rotate keys.
- Upon anyone joining or leaving a key rotates for everybody. New messages use this new key. Clients must keep all past keys locally to see older messages.
Do people migrate all past rotating keys over to new devices?
-
Members of large groups do know each other's identities.
-
There is a admin/moderator to kick people out
-
All the group members have a chance to rotate keys
-
Upon anyone joining or leaving a key rotates for everybody. New messages use this new key. Clients must keep all past keys locally to see older messages.
-
There is also an event chain that can recover all the past keys
There is also an event chain that can recover all the past keys
Who can see this event chain and, if you are rotating keys all the time, how big is it? And how do new users don't see the past if they also have access to the event chain?
It can be achieved through the rotate keys solution. A person who joins group chat can decrypt the rotate events from the point they joined until they leave.
It can be achieved through the rotate keys solution.
How? Will each user keep their list of keys in their own event chain? Is this event signed by a client, with the user's account, or by the moderator to each member?