lilliput
lilliput copied to clipboard
discord
Add support for animated WebP images
This PR adds support for resizing animated WebP images through the libwebpmux/demux sublibraries. It uses the WebPAnimEncoder/WebPAnimDecoder APIs to implement new Encoder/Decoder classes instead of relying on OpenCV, similar to the giflib codec.
This does add an additional dependency on those two libraries, but since they're included with the already required libwebp library (just disabled by default), this shouldn't cause much more hassle beyond rebuilding dependencies.
I've done some testing with animated WebPs I had on hand, as well as making sure that non-animated WebPs go through the normal OpenCV route, but it's probably a good idea to run it with other files as well (I only happen to have lossless files at the moment).
I'm not quite sure what happened with the png headers - I guess they were turned into links when I updated the dependencies? I can revert those changes if necessary, but I expect they'll be reverted the moment someone else updates them again.
Closes #97
Disregard above comments, I realize now that this will be a very different beast than GIFs because of the VP8 content. I think in that respect, animated WEBPs look more like video transcodes. In light of that, I'm happy to bring this functionality to lilliput, but I'd like to gate it behind some kind of feature toggle. Most likely I would guess Discord won't enable this for a while since we are pretty careful about limiting how much work is done per asset in our resizer. However, I'd be happy to have the functionality provided to lilliput! Thank you again for sending the PR.
I think then the remaining blockers would just be to add a toggle similar to gifMaxFrameDimension to allow us to toggle animated webp decoding on or off, and to split the build deps changes into a new PR. Once I have the build deps script change, we can merge it and run our CI job that will check in all the actual deps changes.
Most likely I would guess Discord won't enable this for a while since we are pretty careful about limiting how much work is done per asset in our resizer.
Please, please please PLEASE bring it to Discord. We've been requesting this for YEARS. libwebp has a quality/speed option. Use it.
https://support.discord.com/hc/en-us/community/posts/360034605072-Add-support-for-WEBP-Animated-Images-and-Emojis
Also, fixes https://github.com/discord/lilliput/issues/97
@Zipdox Discord already supports .mov and .mp4 video containers. These would be your best bet for uploading video.
We want to replace GIF with WebP. It's not about posting videos. If you want us to use videos instead of animated images then Discord should add an autoplay (muted audio ofc) and loop option.
@brian-armstrong-discord:
I realize now that this will be a very different beast than GIFs because of the VP8 content. I think in that respect, animated WEBPs look more like video transcodes.
AFAICT, WebP doesn't use any sort of true delta frames, unlike plain VP8 or other video files - it just stores keyframes with "dirty rects" to avoid redrawing the same regions. Google appears to pose it as a list of frames that just use the VP8 bitstream format & basic frame compression methods, but it lacks the delta coding and other advanced stuff usually applied to video. (I might be off base with this one though - I don't have much experience with video codecs.)
Discord already supports .mov and .mp4 video containers. These would be your best bet for uploading video.
The point of using WebP is to replace what GIF does now: small, auto-playing animations without audio that can also be viewed as images. GIF is annoying because the files are much larger compared to other formats, while MP4/MOV don't quite fit the use case as they are often larger, may store audio, and (critically) are not auto-playing.
I prefer using WebP in my app over MP4/MOV/WebM because a) it's not hard to encode at all, b) it's well-supported by basically every browser (minus Discord), and c) it fits the niche that GIF has while being much smaller.
There's been plenty of discussion about this on the Feedback forums and Reddit, though I'm sure you all have browsed through all that a few times.
I'm happy to add any changes you may want, though - I'm currently starting classes & juggling some other software stuff, so I don't have a whole lot of time to work on this right now, but I can work on it when I get the chance.
@brian-armstrong-discord Whether animated WebP ends up being supported or not, something has to be done about this:
Right now it's straight up broken. If Discord won't re-encode animated WebP, it has to do one of the following:
- Pass the original image through unchanged (preferred)
- Show a static image
I realize now that this will be a very different beast than GIFs because of the VP8 content. ... Most likely I would guess Discord won't enable this for a while since we are pretty careful about limiting how much work is done per asset in our resizer. ...
A stopgap for Discord might be to have a multi-layer approach, checking the content of the WEBP RIFF and only transcoding WEBPs that do not contain VP8 or VP8X chunks. Lossless chunk decompression would still be unbounded, but that's a problem you would already see when scaling PNG images.
I got a bit of time to implement the WebP animation enable flag. To avoid the loading symbol that's plagued Discord for years, I've made it so that at the very least the first frame of the animation is resized as if it were a single-frame file. This shouldn't put much more of a burden on the server, as it already handles static WebP through the OpenCV code. This will likely still cause confusion in users when their animated files are suddenly static, but it's far better than the infinite loading wheel today.
but it's far better than the infinite loading wheel today.
Look at my previous comment, now it's just a blank file icon. In my opinion, the files should just be passed through unchanged instead of resizing them. It was like this back in the day, according to old Reddit posts I found. What changed? Anyway, being WebP, the files will be much smaller than GIFs anyway, so they will put less strain on the network.
