QwikDev
Image Optimization
Is your feature request related to a problem?
- Partytown solves 3rd party script slowdown
- Qwik solves main application slowdown
But there's a third and final "slow" thing to load: images
It's not hard to see a world where Qwik apps are basically always loading so fast that the images are the very last thing to cross the finish line. And for UX, it's a very jarring experience to have images load in so slowly. Not to mention many times they are understandably non-negotiable and a core part of the page experience.
Creating a robust solution for faster image loading and better handling of the loading experience -- This seems to align perfectly with BuilderIO's vision, but only so much can be taken on by the core team.
Describe the solution you'd like
This is a perfect opportunity for the pooled effort of the community to build a feature that virtually everyone will benefit from (how many pages have you pushed out to the world that didn't have at least one image on them?)
This is a high-level request, with the only breakdown I'd like to suggest is to split the thoughts between:
- Genuinely improving the load speeds in the first place
- Improving the perception & UX while the image loads
Describe alternatives you've considered
One rough example to think of is NextJS's <Image /> component as it's designed to approach the same problem.
There are also myriad strategies being used out in the wild right now to help with various parts (auto converting images to .webp, etc)
Additional context
Just to add, I can personally take on one aspect of a smoother loading experience: showing a blurred image while loading the full one.
I built a similar component for a Next project because our team was using pure static exports of Next, so their built-in <Image /> was no longer relevant and we needed a custom way to do it
+1 for this, next to Authentication.
( https://github.com/BuilderIO/qwik/discussions/1507 )
Just freeforming a few notes here. @wmertens Doing some awesome work and taking the blur-while-loading approach to a different level with https://github.com/wmertens/qwik-starter-blurhash
There are a TON of clever optimizations we can do between getting the image to load fast, and making the UX feel good during the times where the longer load is inevitable.
Separate but related thought: Image optimization should also extend to making it easy to manage the multi-sized images for different devices/screen sizes. Can't help but think at some point an <Image /> component will be necessary where at build time we generate the appropriate set of different sizes.
And the component itself feels like just using a single <img> component but in reality is a <picture> with the appopriate <source>s + media queries underneath + a blurhash implementation, and the needed logic to smoothly transition in the fully loaded image in place of the placeholder if blurhash was used
Any objections to using the sharp library for the abovementioned generation of different image sizes? I've had success with it in the past, it's pretty popular, but are there any pitfalls? Issues? Or just better libraries for any other reason?
@wmertens Found a great library to use or take lessons from https://plaiceholder.co/ (from this Discord thread )
BTW looks like https://plaiceholder.co/ is the library to use, it uses sharp under the hood and on their demo page they use PNG files for the base64 background I'm more convinced of the base64 css background now, and I suspect that the largest part of the base64 is headers for the PNG file, which are static. So you'd only have to keep the dynamic part of the string in your db and in queries, probably similar in size to blurhash
Ah! Thanks for creating this @KenAKAFrosty! I made this proposal a while ago but I guess the team didn't do anything with it publicly, perhaps we should move it to this issue, another one, or an RFC
I agree completely that easy image optimization through an Image component is critical. We actually have a lot of learnings from this in implementing the frontend component and backend API for this for Builder.io. I can definitely confirm that Sharp works great. I think Next.js mostly nails it and worth taking significant inspiration from
Here is the original proposal I wrote up that sounds like never made it's way to github but worth throwing in here as it basically replicates what was successful with us from Builder.io, combined with experience with Next's image component (plus a couple adjustments based on learnings)
Qwik Image Component
Detailed loom walking through this doc
Goals
The main goal is to make it easy to ensure you can seamlessly use any image in Qwik, but to ensure it is optimized for speed. This means
- Use proper sizes (always show the right size image for the right screen size, using dynamically generated
srcset) - Use next gen formats when possible (e.g. using webp when possible via
<source>) - Use browser-native lazy loading by default, but be able to be turned off for high priority images, such as your LCP image
- Avoid layout shifts (images need a fixed height/width or an aspect ratio so they don’t cause a layout shift when they load)
The main things needed for this are
- A frontend
<Image />component that should feel quite a lot like the plain<img>tag in terms of DX - A backend API in Qwik City to dynamically serve images in different sizes and formats
References/inspiration
Note: Most of this stuff is simple. The tricky part is really the backend/api that can serve the default loader
Responsive Images
This is the best default for most images, to resize to any width
Input:
<Image src="..." aspectRatio="16 / 9" />
Output:
<picture style="display: block">
<source type="image/webp" srcset="
/image?width=100&format=webp 100w,
/image?width=200&format=webp 200w
/image?width=400&format=webp 400w
...">
<img loading="lazy" style="aspect-ratio: 16/9; width: 100%" srcset="
/image?width=100 100w,
/image?width=200 200w,
/image?width=400 400w
" src="/image">
</picture>
Fixed Size Images
Best for things that do not resize, like small icons
Input:
<Image src="..." width={50} height={50} />
Output:
<picture style="width: 50px; height: 50px">
<source type="image/webp" srcset="
/image?width=50&format=webp 1x,
/image?width=100&format=webp 2x
...">
<img loading="lazy" width="50" height="50" srcset="
/image?width=50 1x,
/image?width=100 2x
" src="/image">
</picture>
Image Attributes
Common image attributes, like alt, sizes, etc should be forwarded to the <img> element. But style class etc should go to the <picture>.
We may also want a way to provide specific img props, e.g.
<Image imgProps={{ style: {...}, class: '...'}} src="..." />
That way you can do more advanced things
Priority
When priority="high", do not lazy load. In the future we may want to preload high priority images as well as maybe set fetchpriority="high"
Loaders
Default Loader
So this is the tricky part. By default we should actually host an API from Qwik City that can dynamically compress and cache images
In Builder we have a simple API powered by sharp. So ideally, when you specify
import foo from './foo.png'
// Path to a file
<Image src={foo} ... />
// Direct URL
<Image src={url} ... />
<Image src="https://..." ... />
We instead convert that to hit an internal API to compress images, e.g.
<img srcset="
/_qwik/abc123?width=100&format=webp 100w,
/_qwik/abc123?width=200&format=webp 200w,
...
">
Ideally the images returned are immutable (cached forever), but perhaps there is an attribute to override that
Custom Loaders
Next.js has a good API for this
import Image from 'next/image'
const myLoader = ({ src, width, quality }) => {
return `https://example.com/${src}?w=${width}&q=${quality || 75}`
}
const MyImage = (props) => {
return (
<Image
loader={myLoader}
src="me.png"
alt="Picture of the author"
/>
)
}
Notes
Do not use WEBP or srcset for SVG
<picture>
<img loading="lazy" style="aspect-ratio: 16/9" src="..." >
</picture>
We should probably make the deviceSizes configurable like Next.js somewhere
- Note that WebP is now superceded by AVIF. No point in supporting WebP, since AVIF is better across the board, all browsers are supposed to support it (Edge should be implementing support) and there's always the fallback to the original jpg/png.
- I did some experiments and I think it's possible to generate a 5x5 blurry preview .gif data URI in about 200 bytes. This could be used as the default image in a srcset.
- Note that WebP is now superceded by AVIF. No point in supporting WebP, since AVIF is better across the board, all browsers are supposed to support it (Edge should be implementing support) and there's always the fallback to the original jpg/png.
If AVIF is not supported by the browser then WebP should be served, not the original one.
AVIF is only supported by 76% of market share, and it will take some time to get all those Edge browsers updated to a compatible version.
Better to bet on current features, and not on promises.
https://avif.io/blog/tutorials/edge/ - Microsoft already has av1 support as a plugin. I do think they'll have it soon ish.
In any case, if I have to spend conversion cpu and storage on images, I want to limit it to 2 to types: the absolutely required minimum, and the best. WebP is not the best. I totally understand that you have different trade-offs, so IMHO the Image solution should allow defining what formats and sizes are supported.
Given e.g. a 5MP original jpg, I'd only offer something like:
- 5x5 gif preview data URI
- 60% quality jpeg fitting in 2000x2000
- 75% quality jpeg in 500x500
- AVIF in 1000x1000 + original size
- square versions of the above
If enough sites do this, Edge will notice that AVIF support is needed.
@wmertens Found a great library to use or take lessons from https://plaiceholder.co/ (from this Discord thread )
BTW looks like https://plaiceholder.co/ is the library to use, it uses sharp under the hood and on their demo page they use PNG files for the base64 background I'm more convinced of the base64 css background now, and I suspect that the largest part of the base64 is headers for the PNG file, which are static. So you'd only have to keep the dynamic part of the string in your db and in queries, probably similar in size to blurhash
Hey, I’ve used Plaiceholder before & ended up cancelling immediately after generating a handful of images & realizing the results were inferior.
https://avif.io/blog/tutorials/edge/ - Microsoft already has av1 support as a plugin. I do think they'll have it soon ish.
In any case, if I have to spend conversion cpu and storage on images, I want to limit it to 2 to types: the absolutely required minimum, and the best. WebP is not the best. I totally understand that you have different trade-offs, so IMHO the Image solution should allow defining what formats and sizes are supported.
Given e.g. a 5MP original jpg, I'd only offer something like:
5x5 gif preview data URI
60% quality jpeg fitting in 2000x2000
75% quality jpeg in 500x500
AVIF in 1000x1000 + original size
square versions of the above
If enough sites do this, Edge will notice that AVIF support is needed.
Allow me to point something out: CDN storage is incredibly cheap. Processing is incredibly cheap. Waiting significantly longer for an image-rich ecommerce site to load is very expensive, because it causes users to get frustrated & bail on a purchase. Faster sites result in more sales revenue, and when people do convert, they convert with much larger purchases.
Making users wait for images to load is exactly opposite the core promise of Qwik.
I don't understand the benefit of adding all sorts of restrictions around what size, aspect ratio & encoding type of images that will be supported. If a user has a non-square image, why not support it? Many of these are solved problems, and it's just a question of whether any of them comes with significant disadvantages, such that we might exclude it from defaults or give users the option to opt-out. Instead of saying "this MUST not support [whatevs]" or "this MUST only support [whatevs]", please let's focus on the GOALS and then talk about advantages and disadvantages that come along with each decision.
Achieving product/market fit comes down to smart defaults AND allowing the user to make the choice to depart from these defaults, when they don't work as well for a particular use case. We shouldn't be making an all-or-nothing decision on encoding types. Clients advertise what encoding types they accept in the request header, & it should be our default to provide optimal performance & image quality for as many users as possible. This means using AVIF, WebP, PNG, GIF, even JPEG & SVG, depending on the type of image (e.g. very few colors, animation, etc).
Yes, it takes a little longer to encode more than only doing 1 or 2 formats, but it shouldn't be our job to say "too bad, you're using Edge, so let's convince Microsoft to fix their AVIF support by degrading performance even further".
IMO, top priority should be performance & image quality, for as many users as possible.
If users want to make compromises on one or more of these priorities, let them choose to override the defaults. If they do nothing, out of the box, they get a great DX & their users get a great experience.
In the case of on-demand optimization, we could allow users to start with a lazy approach to generating all formats, such that only the formats & sizes requested would be generated & CDN'd. But again, storage & processing is cheap. IF you break it up into a lazy approach, then you end up waiting to load the highest resolution image into memory multiple times, which inevitably costs more in both cloud compute and more users waiting an inordinate amount of time.
I'd much rather load once, process once (eagerly) for "all the things", then store the result until the end of time. This way, you only pay the price to load the image for processing once, and after that point, any user that needs to load the image into an app or site built with Qwik can do so without any additional delay.
Hmm, good points. Some years ago an image hosting solution was actually vastly more than the price of hosting the website, and we instead developed an on-demand transcoding middleware that just serves from the webserver. Of course this means paying attention to the size of the hosted images. I see that now CDN hosting is a fraction of the price, and your reasoning makes a lot more sense.
In any case, I do think that we cannot mandate a CDN. So the dev must be able to make decisions around this. The Image component should work always, SSG or SSR, simple http file server or CDN network.
One more thing: I just read up on picture vs img and I think Image should produce img. picture is for art direction and that would need to be another component, or maybe Image with complex attributes.
One more thing: I just read up on picture vs img and I think Image should produce img. picture is for art direction and that would need to be another component, or maybe Image with complex attributes.
I agree @wmertens 🙏🏼
@zanettin I think we can close this issue In this PR #2860 we decided to create a third party library https://github.com/qwikifiers/qwik-image
