High Time to First Byte (TTFB)

[marco910]

Before opening, please confirm:

• [X] I have checked to see if my question is addressed in the FAQ.
• [X] I have searched for duplicate or closed issues.
• [X] I have read the guide for submitting bug reports.
• [X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

App Id

d33j1egj9kkcu2

Region

eu-central-1

Amplify Hosting feature

Performance

Describe the bug

I deployed a Next.js application with SSR to Amplify Hosting. When I request the page in the browser, I'm getting a high Time to First Byte (TTFB) on the initial request. All other resources like CSS and JS are loading normally and quick.

When I check this on multiple devices, I'm getting always the same result with a high TTFB.

With "high TTFB" I mean a TTFB of +1s and a complete page load time of +3s.

Expected behavior

The TTF shouldn't take that long. When I host the application on a custom Node.js Server is way faster.

Reproduction steps

1. Deploy a Next.js app with SSR to Amplify in the "eu-central-1" region.
2. Open it up in the browser and have a look in the "Network" tab of the dev tools in your browser.

Build Settings

No response

Additional information

No response

[ghost]

Hi @marco910 👋🏽 thanks for opening this issue. I'm not able to reproduce this but i can see this behavior happening with your app. I will update once I have more information.

[marco910]

Hi @hloriana Thank you for your quick reply! I think the problem is maybe located between CloudFront and S3/Lambda. Because when I request the site on one device and few seconds later on another device it works very well and without the high TTFB. And then, if I wait a few minutes and refresh the page, I'm facing the same problem.

[marco910]

Hi @hloriana Quick update for you: Today, I deployed a new app with the ID dwlqy3pibcqzt to the Amplify Console. The app runs also on Next.js with SSR and I'm facing the same issue as with the other site. When I have a look in the response header of the site, I'm getting two headers of x-amz-cf-pop with two different values. Is that normal or a bug?

[ghost]

Was this new app a basic SSR app? Are you using any specific SSR features by any chance?

[jedwardblack]

I am experiencing the same issue with my NextJS app with SSR on AWS Amplify. 12 seconds TTFB for this site: https://getyourpiece.io Occasionally, I can get a sub 1 second TTFB which leads me to believe it has something to do with caching, but I am not sure. Thanks for your help!

[marco910]

@hloriana Yes, it's a basic Next.js app with getStaticProps() and Incremental Static Regeneration. All the pages are built at build time or in the background. So, there is nothing that could cause the high TTFB. There are no "special" features. When I deploy the site to custom server, it's really fast. I'm also deploying a Gatsby site with Amplify Hosting. With that, I don't have problems.

@jedwardblack In the response headers of your site saw that you are also getting the double x-amz-cf-pop header. I agree with your guess that this could be caused by caching or CloudFront.

[jedwardblack]

I looked up the x-amz-cf-pop location codes that are in the response headers for my site using this: https://www.feitsui.com/en/article/3

The IAD50-C2 location is Washington Dulles; that makes sense. The DWF56-P1 location doesn't exist, at least from what I could find. The other Dallas-Fort Worth locations do exist, but not P1.

It is probably a problem that there are 2 of those headers, but I would imagine that the bigger problem is that the DFW56-P1 location doesn't exist.

@marco910 What are the two x-amz-cf-pop location codes you are getting in your response headers?

[marco910]

@jedwardblack That depends on the location you're requesting the site from. But I get for the document itself these headers:

• FRA56-C1: Frankfurt, Germany
• LHR61-C2: London, UK

They both exist in the list. I'm only experiencing this with Next.js SSR sites and not with my Gatsby site.

[marco910]

Hi @hloriana Are there any updates on this?

[tyfuji]

Hi @hloriana we are experiencing the same problem with a SSR Next.js app. TTFB is regularly 300ms+ on all resources. We are testing using the provided amplify domain, not a custom domain, so the domain shouldn’t be a factor. Can anyone help with this?

[tyfuji]

To add some more information:

• The high response times are always accompanied by a "x-cache: Miss from Cloudfront" response header.
• Reloading the page immediately is fast (within 2 seconds) and all the resources Hit from Cloudfront.
• Loading a static resource in a new tab, we get Hit from Cloudfront and the resources are fast

The default Amplify Cloudfront TTL policy seems to be 2 seconds, could that have something to do with it?

[thibauld]

Hi @hloriana :wave: Same issue here.

We have a Next.js SSR app hosted on Amplify with the same High TTFB issue for the text/html content (no issue for other type of content as far as I can see).

Our app id is d3pufjj6b49muy on us-east-1

Here's more context:

• App is deployed from production branch where performance mode is enabled

• From your doc, it seems that performance mode should activate some edge CDN caching:

Amplify Hosting performance mode optimizes for faster hosting performance by keeping content cached at the edge of the CDN for a longer interval.

• But it doesn't as we can see cache-control: max-age=0 being passed in the request headers:

• Which logically causes Cloudfront to NOT cache the response:

• I thought that a quick fix would be to set custom headers...

• ... but it didn't work for a reason that is unknown to me :man_shrugging: It seems like a bug if I judge by #1170 and #2281

Happy to share more information if that can be helpful.

[marco910]

Hi @thibauld I'm facing the same issue as you on two different Amplify apps.

One app has a config for custom headers that looks like this:

customHeaders:
  - pattern: /_next/static/**/*
    headers:
      - key: Cache-Control
        value: 'public, max-age=31536000, immutable'
      - key: x-frame-options
        value: sameorigin
      - key: x-xss-protection
        value: 1; mode=block
  - pattern: /_next/data/**/*
    headers:
      - key: Cache-Control
        value: 'public, max-age=0, s-maxage=0, must-revalidate'
      - key: x-frame-options
        value: sameorigin
      - key: x-xss-protection
        value: 1; mode=block
  - pattern: '**/*[.html]'
    headers:
      - key: Cache-Control
        value: 'public, max-age=0, s-maxage=0, must-revalidate'
      - key: x-frame-options
        value: sameorigin
      - key: x-xss-protection
        value: 1; mode=block

All headers are getting applied and hitting from CloudFront if the files are cached. But I'm also facing the problem with the TTFB on the document request, which is around 3 seconds.

On the second app I'm only using performance mode without any custom headers and having a total page load time of around 6 seconds which is way too much.

[ghost]

@marco910 apologies for the delay. Can you provide the other app ID for the app that's experiencing this issue? @thibauld thanks for providing these details. They will definitely help us determine the root cause. @tyfuji can you provide your app ID as well?

[marco910]

@hloriana The second site where I'm facing the issue is dwlqy3pibcqzt

[ken-nah]

Hi @hloriana I am experiencing a similar issue as well.

[tyfuji]

@hloriana Our app ID is dpbvd6d215akw thanks!

[tyfuji]

We actually found some important information on this: It seems to be linked in our case to a cookie that gets modified on each pageload. It appears that changes to a browser cookie invalidate the Cloudfront cache, which by default this should not be the case. When we load our resources in fresh tabs, they become hits in Cloudfront, and future refreshes continue to stay hit, and have low TTFB. It is the browser environment which causes the cookie to continue changing, and therefore invalidates the Cloudfront cache.

We looked for the right Cloudfront distro in our AWS console but can’t find it – it appears that Amplify Cloudfront distros are not manageable in AWS console but are configured by default. So I think that is the default config we would have to modify.

[tyfuji]

Hi there, any update on this @hloriana ? Unfortunately our client is unable to switch to Amplify because of this issue.

[richmengsix]

Any update on this issue? Our website takes 7 seconds to load after switching to SSR..

[TheTapHouse]

@hloriana We are also having this issue, anything we can do to mitigate in the meantime?

[ghost]

Hi everyone, we apologize for the delay, we are prioritizing this issue and are working on reproduction and a possible workaround. Thank you all for your patience!

[Nimdy]

same. Just wanted to point it out.

[tyfuji]

Thank you @hloriana . Please keep us updated. We decided to try a dummy test app with SSG instead of SSR and got the same problem. This app shows "Next.js - SSG" as the framework in Amplify.

https://www.loom.com/share/cd753b01a5e542d29985f1e14abb0016

[tyfuji]

This has become a significant business problem for us unfortunately. If we could have even some kind of hacky workaround in the short term it would be a big help.

[jedwardblack]

@tyfuji I was only able to resolve this issue by moving my frontend to Vercel. It builds and deploys in under 3 minutes, and the backend lives happily in Amplify. Now I am seeing TTFB under 300ms.

The occasional cold-start was still an issue. So, I set up a Lambda that pings my site every 10 minutes. Problem solved.

All of that to say, I love Amplify. It has helped me make more progress in the past 3 months than many years prior. I am excited to see it move forward, but for now this was my best option.

[tyfuji]

Thanks @jedwardblack ! Not an option for us but appreciate the tip.

@hloriana We were able to do some testing in our apps with performance mode on and off, and testing both static resources and the base index file. It seems like performance mode does more than just affect the headers. The basic finding is that the s-maxage and maxage directives do not influence the cache behavior the way one would expect, and that both SSG/SSR and Performance mode may be having an influence outside of just the headers.

On SSG you can get documents to cache, and on SSR you can get static assets to cache, but there doesn’t appear to be a solution where you can get both to cache at once.

Unfortunately the method for getting static assets to cache on SSR is to set the maxage directive; this is not ideal because it also affects browser caches. We would much prefer to be able to influence the browser cache through the s-maxage directive.

Next.js SSG (d1q37v148cianr)

Performance Mode Off

Static resource

• Cache-Control: public, max-age=0, immutable, s-maxage=2
• Result: ❌ Regularly getting slow “Refreshhit”, but refreshing in under 1s results in a fast “Hit”

Document

• Cache-control: public, max-age=0, s-maxage=2, must-revalidate
• Result: ❌ Regularly getting slow “Miss”, but refreshing in under 1s results in a fast “Hit”

Performance Mode On

Static Resource

• Cache-control: public, max-age=0, immutable, s-maxage=600
• Result: ❌ Regularly getting slow “RefreshHit”, but refreshing in under 1s results in a fast “Hit”
• Experiment:
  • Changed cache-control to: public, max-age=31536000, immutable
  • Result: ❌ Regularly getting slow “RefreshHit”, but refreshing in under 1s results in a fast “Hit”

Document

• Cache-control: public, max-age=0, s-maxage=600, must-revalidate
• Result: ✅ All good!

Next.js SSR (dpbvd6d215akw)

Performance Mode Off

Static Resource

• Cache-Control: public,max-age=31536000,immutable
• Result: ✅ It caches!!

Document

• Cache-Control: public, max-age=0, s-maxage=2, must-revalidate
• Result: ❌ Regularly getting slow “Miss”, but refreshing in under 1s results in a fast “Hit”

Performance Mode On

Static resource

• Cache-Control: public, max-age=0, immutable, s-maxage=600
• Result: ❌ Regularly getting slow “Miss”, but refreshing in under 1s results in a fast “Hit”
• Experiment:
  • Changed cache-control to: public,max-age=31536000,immutable
  • Result: ✅ It caches!!!

Document

• cache-control: public, max-age=0, s-maxage=600, must-revalidate
• Result: ❌ Regularly getting slow “Miss”, but refreshing in under 1s results in a fast “Hit”
• Experiment:
  • Changed cache-control to: public, max-age=0, s-maxage=600 (Removing must-revalidate)
  • Result: ❌ Regularly getting slow “Miss”, but refreshing in under 1s results in a fast “Hit”

[tyfuji]

Just checking on this! For now we have had to set up production hosting off of amplify until this is resolved. We’d love to fully consolidate into Amplify as soon sa we can.

[marco910]

Hi guys! This issue exists now already 4 months in Amplify Hosting... Are there any updates on this?

[Nimdy]

I seem to have adjusted a lot but its hit or miss. I have a cheat but its working, I need to get it working better.