nips icon indicating copy to clipboard operation
nips copied to clipboard

Idea: Save hash of external media in the event

Open jonas-lundqvist opened this issue 2 years ago • 11 comments
trafficstars

I posted this on reddit, but I don't know if it reached the relevant people there so here is a copy-paste:

Since external media (primarily images) are stored externally and Nostr notes just contain the URL users are relying on the host of said media to always be online and not modify the content. One mitigation for users to verify that whatever image someone posted in a note is immutable would be to save the hash of it within the note.

I propose to handle this as such: An URL to an image inside .content should have the form [<url> | <hash>] where <hash> is a SHA256 hexencoded string of the raw image data.

Example:

{
  "content": "Nostr in sheer decline: [ https://asdf.com/89asdf.gif | 396c4a297752a82a080f61fb8a4614c5ff3bef0f666830e70276a4f3465d2597 ]\n\nIt was all a dream.",
  "created_at": 1677184528,
  "id": "<id>",
  "kind": 1,
  "pubkey": "3bf0c63fcb93463407af97a5e5ee64fa883d107ef9e558472c4eb9aaaefa459d",
  "sig": "<sig>",
  "tags": [],
  "relays": [
    "wss://nostr.mom"
  ]
}

Old clients can still just post the URL and clients aware of this NIP could display it with an "unverified" notification or such. Once clients starts to post images in the new format old clients will display some "garbage" data. I see this being a rather small change and would like some feedback from clients.

If anything similar has been proposed (and rejected) please let me know.

jonas-lundqvist avatar Feb 25 '23 10:02 jonas-lundqvist

Not sure about the specifics of how and where to include such a hash, but strongly support the concept.

In fact, I would rather like to see all media links directly addressed by their hash, then perhaps another replaceable event could resolve that hash to a URL. That way, if some host goes down, only the URL event needs to be replaced to update the URL, the original event stays intact.

shafemtol avatar Feb 25 '23 12:02 shafemtol

Another thing to consider: To account for media that can be streamed or otherwise viewed before fully downloaded, one may want to use a Merkle root rather than a plain hash. That way, if given either the full list of leaf hashes or a Merkle proof on parts of the leaf hashes, a client can verify the downloaded parts before viewing them.

Content type should perhaps also be explicitly stated.

shafemtol avatar Feb 25 '23 12:02 shafemtol

The best way to do it would be to add tags to the event containing the URL hashes.

fiatjaf avatar Feb 25 '23 18:02 fiatjaf

It's a good idea as a tag.

mikedilger avatar Feb 28 '23 04:02 mikedilger

How will you know the hash of the file if the service is processing images after upload?

v0l avatar Feb 28 '23 10:02 v0l

How will you know the hash of the file if the service is processing images after upload?

Client can download the uploaded file and show the result back to the user to review before signing.

The server could change the file it serves at some later point. The user (or possibly anyone) should then have the option to upload the original file somewhere else and add it as an alternative URL through a replaceable event. Another option might be for the user (not someone else in this case) to add the new hash as valid through a replaceable event.

shafemtol avatar Feb 28 '23 11:02 shafemtol

How will you know the hash of the file if the service is processing images after upload?

Good point.

Another question is: is anyone going to implement this in all seriousness? How does it help? I think if you want real decentralization for media you should use torrent magnet links with an optional "web seed" fallback (nostr.build, please implement that!). The torrent protocol already checks hashes by default.

fiatjaf avatar Feb 28 '23 11:02 fiatjaf

How will you know the hash of the file if the service is processing images after upload?

Well i just ask this because i specifically return the hash of the post-processed file on void.cat, but i dont know how possible this will be for other general file stores like S3 buckets etc.

Another question is: is anyone going to implement this in all seriousness?

I didnt look at webtorrents in a long time i though they are really slow?

v0l avatar Feb 28 '23 14:02 v0l

It would be great for media attachments to have a tag, like "m". Then it could support valuable metadata such as mimetype, dimensions or duration, blurhash, and other queryable data that is hard for clients to find without fetching the attachment.

Also, something visual like blurhash (or a similar algorithm) for images could potentially validate the content and improve the way it's displayed visually.

alexgleason avatar Mar 01 '23 01:03 alexgleason

+1 for blurhash, but seems unrelated to this issue, maybe you should create another issue for media metadata

v0l avatar Mar 01 '23 11:03 v0l

Maybe this: https://nostr.guru/nevent1qqsxgxcsq5vevy4wdty5z5v88nhwp2fc5qgl0ws5rmamn6z72hwv3qcpz3mhxue69uhhyetvv9ujuerpd46hxtnfdu0cu4mu

fiatjaf avatar Mar 01 '23 13:03 fiatjaf