Amplify not honoring Incremental Static Regeneration due to sqs.fifo deduplication

[jayfry1077]

Before opening, please confirm:

• [X] I have searched for duplicate or closed issues and discussions.
• [X] I have read the guide for submitting bug reports.
• [X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

JavaScript Framework

Next.js

Amplify APIs

DataStore

Amplify Categories

storage, hosting

Environment information

# Put output below this line
  System:
    OS: Windows 10 10.0.22000
    CPU: (12) x64 AMD Ryzen 5 3600 6-Core Processor
    Memory: 8.38 GB / 15.93 GB
  Binaries:
    Node: 14.15.1 - C:\Program Files\nodejs\node.EXE
    npm: 6.14.8 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Spartan (44.22000.120.0), Chromium (97.0.1072.55)
    Internet Explorer: 11.0.22000.120
  npmPackages:
    @ampproject/toolbox-optimizer:  undefined ()
    @aws-amplify/datastore: ^3.7.3 => 3.7.3
    @babel/core:  undefined ()
    @peculiar/webcrypto:  undefined ()
    @tailwindcss/typography: ^0.5.0 => 0.5.0
    @vercel/nft:  undefined ()
    amphtml-validator:  undefined ()
    arg:  undefined ()
    async-retry:  undefined ()
    async-sema:  undefined ()
    autoprefixer: ^10.4.1 => 10.4.1
    babel-packages:  undefined ()
    ci-info:  undefined ()
    cli-select:  undefined ()
    comment-json:  undefined ()
    compression:  undefined ()
    conf:  undefined ()
    content-disposition:  undefined ()
    content-type:  undefined ()
    cookie:  undefined ()
    cross-spawn:  undefined ()
    date-fns: ^2.28.0 => 2.28.0
    debug:  undefined ()
    devalue:  undefined ()
    escape-string-regexp:  undefined ()
    find-cache-dir:  undefined ()
    find-up:  undefined ()
    formdata-node:  undefined ()
    fresh:  undefined ()
    glob:  undefined ()
    gzip-size:  undefined ()
    http-proxy:  undefined ()
    icss-utils:  undefined ()
    ignore-loader:  undefined ()
    is-animated:  undefined ()
    is-docker:  undefined ()
    is-wsl:  undefined ()
    json5:  undefined ()
    jsonwebtoken:  undefined ()
    loader-utils:  undefined ()
    lodash.curry:  undefined ()
    lru-cache:  undefined ()
    micromatch:  undefined ()
    mini-css-extract-plugin:  undefined ()
    nanoid:  undefined ()
    native-url:  undefined ()
    neo-async:  undefined ()
    next: latest => 12.0.7
    node-libs-browser:  undefined ()
    npm-check-updates: ^12.0.5 => 12.1.0
    ora:  undefined ()
    postcss: ^8.4.5 => 8.4.5 (8.2.15)
    postcss-flexbugs-fixes:  undefined ()
    postcss-modules-extract-imports:  undefined ()
    postcss-modules-local-by-default:  undefined ()
    postcss-modules-scope:  undefined ()
    postcss-modules-values:  undefined ()
    postcss-preset-env:  undefined ()
    postcss-safe-parser:  undefined ()
    postcss-scss:  undefined ()
    postcss-value-parser:  undefined ()
    react: 17.0.2 => 17.0.2
    react-dom: 17.0.2 => 17.0.2
    react-markdown: ^7.1.1 => 7.1.2
    react-server-dom-webpack:  undefined ()
    rehype-highlight: ^5.0.2 => 5.0.2
    rehype-raw: ^6.1.1 => 6.1.1
    remark: ^14.0.2 => 14.0.2
    remark-gfm: ^3.0.1 => 3.0.1
    remark-html: ^15.0.1 => 15.0.1
    remark-images: ^3.1.0 => 3.1.0
    resolve-url-loader:  undefined ()
    sass-loader:  undefined ()
    schema-utils:  undefined ()
    semver:  undefined ()
    send:  undefined ()
    source-map:  undefined ()
    string-hash:  undefined ()
    strip-ansi:  undefined ()
    tailwindcss: ^3.0.8 => 3.0.9
    terser:  undefined ()
    text-table:  undefined ()
    to-vfile: ^7.2.2 => 7.2.3
    ua-parser-js:  undefined ()
    unistore:  undefined ()
    uuid:  undefined ()
    web-streams-polyfill:  undefined ()
    web-vitals:  undefined ()
    webpack:  undefined ()
    webpack-sources:  undefined ()
    ws:  undefined ()
  npmGlobalPackages:
    @aws-amplify/cli: 7.6.5
    cross-env: 7.0.3
    jest: 26.6.3
    nodemon: 2.0.7
    serverless-export-env: 1.4.4
    serverless: 2.48.1

Describe the bug

Issue

Next.js Incremental Static Regeneration (ISR) will periodically request a new version of a page on a website. Behind the scenes on Amplify this appears to create a message in an sqs.fifo queue.

This queue has a deduplication setting on message group. I'm not sure how this group ID is generated, but it appears to be the same across browsers. Interestingly the message deduplication id is different across browsers.

Because of this, after I update data in amplify studio, a new visitor may or may not generate a new page. It depends on if they visited within this 5 minute deduplication window. If you had a spelling error you wanted to fix quickly, you're going to have to wait at least 5 minutes from the last time your page was generated.

This 5 minutes is from the SQS 5-minute deduplication interval. (https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/using-messagededuplicationid-property.html)

Expected behavior

ISR should generate a new web page

Reproduction steps

1. Launch a next.js website using ISR on amplify using amplify datastore and amplify studio
2. Make a new content entry
3. Refresh your website twice to see that new content entry (once to kick off isr, another to view the new static page)
4. Make an update to that same content in amplify studio
5. Refresh your page again, and notice that your content will not update for at least 5 minutes.

Code Snippet

// Put your code below this line.

Log output

// Put your logs below this line

aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

No response

Mobile Operating System

No response

Mobile Browser

No response

Mobile Browser Version

No response

Additional information and screenshots

No response

[chrisbonifacio]

Hi @jayfry1077 👋 Thank you for raising this issue. It seems this is more related to hosting than the JS library so I'm going to transfer this over to the console repo for better support.

[github-actions[bot]]

Hi :wave:, thanks for opening! While we look into this...

If this issue is related to custom domains, be sure to check the custom domains troubleshooting guide to see if that helps. Also, there is a more general troubleshooting FAQ that may be helpful for other questions.

Lastly, please make sure you've specified the App ID and Region in the issue!

[ghost]

Hi @jayfry1077 👋🏽 thank you for raising this issue and providing reproduction steps. I will try to reproduce this behavior and update once I have more information.

[jayfry1077]

@hloriana

Sounds good. Sorry the first step is "launch a next.js website with ISR". If you want I can make a video showing off the behavior and share it if that will help.

[lardyNiji]

Hi, Any news on this, perhaps a work around while waiting some fixes ?

[picosam]

Hello, any news on this please?

[realtebo]

Hello, any news on this please?