MSC2448: Using BlurHash as a Placeholder for Matrix Media

[anoadragon453]

Rendered

Implementations (and iterations):

• Ruma
• matrix-media-repo
• Nheko
• Element Web
• https://github.com/matrix-org/matrix-react-sdk/pull/5099
• https://github.com/turt2live/matrix-media-repo/pull/327
• https://github.com/turt2live/matrix-media-repo/pull/328
• https://github.com/turt2live/matrix-sticker-manager/pull/22
• https://github.com/turt2live/matrix-dimension/pull/420

FCP status comment

[anoadragon453]

All review comments addressed.

[Half-Shot]

I love this! This is awesome. I want it now!

[turt2live]

I've been a bit eager and have implemented this (with a possible download route) here:

• https://github.com/turt2live/matrix-media-repo/commit/bf4fe206a677a064fff1ad9c478862a02b825869
• https://github.com/turt2live/matrix-media-repo/commit/e4596e217104db1e9252ce4d669d1409c8a4ad5a
• https://github.com/turt2live/matrix-media-repo/commit/1879bb166928886f339d8ad248e468ee65428e24
• https://github.com/turt2live/matrix-media-repo/commit/29bdbc5d6fb04f7314b5e7907f5db0ed660bc1b7
• https://github.com/turt2live/matrix-media-repo/commit/120d6cfcc619b89ca0a087fe721b0b7616fef890

[deepbluev7]

I've made an experimental implementation of this for nheko. It notably differs by putting the blurhash key under info. Also it is quite slow, since I wrote a custom encoder/decoder and I didn't optimize it yet. The decode part is also still synchronous. Builds should go up soon in #nheko-nightlies:neko.dev with a bit of luck.

I also only implemented the part that is independent of the media repo for now. I don't try to request a backup hash, I only use the one provided in the event and send them in the event.

[anoadragon453]

Thanks for the implementations everyone! I'll respond and implement the changes you've mentioned now.

I will say I will be quite busy this week, but I'll do my best to be responsive and keep the momentum going.

[turt2live]

This also now has an element-web implementation: https://github.com/matrix-org/matrix-react-sdk/pull/5099

[Sorunome]

mx-puppet-bridge now supports sending these

[anoadragon453]

Thanks for the feedback everyone. I'll update the MSC soon with the remaining points and pop it onto the SCT backlog.

[kegsay]

I like this and it's easy to implement - will get to work on a Dendrite impl (though I'm reluctant to add the download endpoint)

[anoadragon453]

Hello everyone. I've made quite a few updates to this MSC tonight. This includes removing the controversial /_matrix/media/r0/blurhash endpoint and other various fixes.

In addition, I've added BlurHash support to various bits other than m.room.message:

• m.sticker: A blurred image appears while a sticker loads
• m.room.avatar: A blurred image appears for room avatars while looking at the public rooms list (or your client's room list)
• m.room.member: Avatars in a room timeline will slowly load in, instead of a vertical line of identical, blank circles (or no circles).
• URL previews: URL preview images will not be blank while they load.

As a user I'd like to see all of these, and think they'd greatly help improve the feel of the app. I understand though that I've added a lot more words, which people need to read.

I look forward to any more feedback, especially technical, as inline comments on the relevant lines of the MSC.

[anoadragon453]

Given that there have been no adverse reaction to adding blurhashes in a few new places, and that blurhash implementations currently exist in the Matrix ecosystem, I think this is ready for FCP.

@mscbot fcp merge

[mscbot]

This FCP proposal has been cancelled by https://github.com/matrix-org/matrix-spec-proposals/pull/2448#issuecomment-1865356251.

Team member @mscbot has proposed to merge this. The next step is review by the rest of the tagged people:

• [ ] @dbkr
• [x] @uhoreg
• [x] @turt2live
• [ ] @ara4n
• [x] @anoadragon453
• [x] @richvdh
• [x] @erikjohnston
• [x] @KitsuneRal

Concerns:

• ~~Do we do anything with image dimensions in m.room.avatar events?~~
• there doesn't appear to be a server implementation for the profile endpoints

Once at least 75% of reviewers approve (and there are no outstanding concerns), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for information about what commands tagged team members can give me.

[KitsuneRal]

@mscbot concern Do we do anything with image dimensions in m.room.avatar events?

[t3chguy]

Bunch of PoCs:

https://github.com/matrix-org/matrix-react-sdk/pull/5099 https://github.com/turt2live/matrix-media-repo/pull/327 https://github.com/turt2live/matrix-media-repo/pull/328 https://github.com/turt2live/matrix-sticker-manager/pull/22 https://github.com/turt2live/matrix-dimension/pull/420

[KitsuneRal]

@mscbot resolve Do we do anything with image dimensions in m.room.avatar events?

[t3chguy]

Slight hurdle, blurhash doesn't support transparencies so doesn't seem appropriate for Stickers

The reference site breaks when given a transparency

https://github.com/woltapp/blurhash/issues/100

[anoadragon453]

@t3chguy please use comment threads.

[turt2live]

@mscbot concern there doesn't appear to be a server implementation for the profile endpoints

edit: or for membership events. Given their fiddlyness in the past, I'd want to see it working in practice first.

[erikjohnston]

@anoadragon453 (or anyone) do you have any bandwidth to make the changes to the MSC? We should land the stuff that is already implemented and in use at the very least

[turt2live]

we're about to hit the 2 year mark since this could have entered FCP, but it's still missing implementations and needs cleaning up. I'm cancelling FCP to bring this back to the near-FCP state, sorry.

@mscbot fcp cancel