GatsbyImage process the image on build & build is slow

[iyyappan-axelerant]

Preliminary Checks

• [X] This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues
• [X] This issue is not a question, feature request, RFC, or anything other than a bug report directly related to Gatsby. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions

Description

As per documentation, I am using Drupal Backend for pulling data. I am using GatsbyImage in graphql query to get the image data. Due to gatsbyImage build is showing IMAGE CDN jobs running & it took around 30 mins to build the 277 pages on server. Server has 2 CPU & 3 GB RAM. Local machine took less time because it has 10 CPU & 16 GB RAM. Added screenshot below.

Can any one tell me why GatsbyImage is processing the image on build? Note: I am not using Gatsby Cloud or Netlify. I am using platformsh server to deploy Gatsby site.

Can anyone tell me how much CPU needed for around 1000 page website & we have lot of images which around 10000 images. ?

Thanks

Reproduction Link

https://github.com/gatsbyjs/gatsby

Steps to Reproduce

Use GatsbyImage in query.

Expected Result

Build time should be less

Actual Result

Build time should be less

Environment

System:
    OS: macOS 13.3
    CPU: (10) arm64 Apple M2 Pro
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.5.1 - /opt/homebrew/bin/node
    npm: 9.8.1 - /opt/homebrew/bin/npm
  Browsers:
    Chrome: 120.0.6099.129
    Safari: 16.4
  npmPackages:
    gatsby: ^5.12.4 => 5.12.9 
    gatsby-cli: ^5.12.1 => 5.12.4 
    gatsby-plugin-config: ^1.0.7 => 1.0.7 
    gatsby-plugin-flexsearch: ^1.0.3 => 1.0.3 
    gatsby-plugin-image: ^3.13.0 => 3.13.0 
    gatsby-plugin-local-search: ^2.0.1 => 2.0.1 
    gatsby-plugin-manifest: ^5.12.0 => 5.12.3 
    gatsby-plugin-mdx: ^5.12.0 => 5.12.3 
    gatsby-plugin-offline: ^6.12.0 => 6.12.3 
    gatsby-plugin-postcss: ^6.12.0 => 6.12.0 
    gatsby-plugin-react-helmet: ^6.12.0 => 6.12.0 
    gatsby-plugin-react-svg: ^3.3.0 => 3.3.0 
    gatsby-plugin-sass: ^6.12.1 => 6.12.3 
    gatsby-plugin-sharp: ^5.13.0 => 5.13.0 
    gatsby-source-drupal: ^6.12.0 => 6.12.3 
    gatsby-source-filesystem: ^5.13.0 => 5.13.0 
    gatsby-transformer-sharp: ^5.13.0 => 5.13.0 
  npmGlobalPackages:
    gatsby-cli: 5.12.1

Config Flags

No response

[iyyappan-axelerant]

Does Image CDN will only works on Gatsby Cloud? Not like normal hosting platforms.

[pieh]

Does Image CDN will only works on Gatsby Cloud? Not like normal hosting platforms.

It does work now on Netlify properly using Netlify Image CDN (without extra hacks) and you can "implement" your own as well - https://www.gatsbyjs.com/docs/reference/release-notes/v5.13/#custom-image-and-file-cdn-url-generators

Please note that above thing is just to generate url for images and you (or your image cdn backend) are responsible for handling those requests. The arguments for transformations are pretty much subset of what Imgix handles so in my tests I did try Imgix backend:

const ImgixClient = require("@imgix/js-core");

const client = new ImgixClient({
  domain: "none-9083.imgix.net",
  secureURLToken: "<redacted>",
});

/**
 * @type {import("gatsby").ImageCdnUrlGeneratorFn}
 */
exports.default = (source, args, pathPrefix) => {
  const url = client.buildURL(source.url, args);

  console.log(`image-cdn`, {
    source,
    args,
    url,
  });

  return url;
};

Just I did have imgix account from some time ago with "Web proxy" source setup (it allows any remote urls) - now that source is enterprise-only in imgix, but there is "Web folder" where you specify "base url" (which would include a domain) and would need to adjust url builder to match expected format.

Alternatively any other image cdn provider could be used as long as it supports at least transformations that you are using in your site.

I did try creating a function/lambda to handle those, but in a lot of cases it was quite problematic, with requests often erroring out due to timeouts etc, so trying to support it this way (where users/platforms don't have to configure image cdns themselves) was a problem magnet and issues caused by this were impacting visitors of the site (while slow build times are annoying, but don't impact site once it's deployed - so out of the 2 it's better to have slow builds than problems on deployed sites)

[pieh]

If you don't want to use ready-made image cdn backends, maybe you could look into https://github.com/unjs/ipx (tho this one out of the box doesn't support remote images - https://github.com/netlify/netlify-ipx is netlify wrapper for it that does support external urls), but this is much more involved and not something you could just "turn on"

In my tests with it I never fully finished proper function with all the functionality - just enough to get something going, but seeing issues I dropped the idea of finishing it up and adding it as part of core gatsby functionality due to issues with that approach (as mentioned in previous comment)

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[github-actions[bot]]

Hey again!

It’s been 60 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it. Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to comment on this issue or create a new one if you need anything else. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community! 💪💜