chromium icon indicating copy to clipboard operation
chromium copied to clipboard

[BUG] No longer working in Lambda

Open fire015 opened this issue 9 months ago • 4 comments

package.json:

    "@sparticuz/chromium": "123.0.1",
    "puppeteer-core": "22.6.4"

code:

  chromium.setGraphicsMode = false;

  console.log("## Launching browser");
  const browser = await puppeteer.launch({
    args: chromium.args,
    defaultViewport: chromium.defaultViewport,
    executablePath: await chromium.executablePath(),
    headless: chromium.headless,
    ignoreHTTPSErrors: true,
  });

  console.log("## Creating new page");
  const page = await browser.newPage();

  console.log("## Setting user agent");
  await page.setUserAgent("Lambda PDF");

  console.log("## Going to " + url);
  await page.goto(url, {
    waitUntil: ["domcontentloaded", "networkidle0"],
  });

It's freezing after "Creating new page" appears in the console logs.

Lambda 18.x (tried with 20.x too) with 2GB memory Runtime version ARN: arn:aws:lambda:eu-west-2::runtime:b475b23763329123d9e6f79f51886d0e1054f727f5b90ec945fcb2a3ec09afdd

fire015 avatar Apr 30 '24 12:04 fire015

Downgrading to:

    "@sparticuz/chromium": "122.0.0",
    "puppeteer-core": "22.5.0"

Seems to have worked. Leaving this open just to verify I didnt make a mistake selecting the wrong package versions?

fire015 avatar Apr 30 '24 13:04 fire015

This issue hit us today but we've not updated for a while so it seems to effect old versions too. Is there anyway to make this project dump a stacktrace when the browser crashes?

leepa avatar Apr 30 '24 15:04 leepa

Make sure to leave graphics mode enabled. The option will be fixed in the next version. I've been traveling so I haven't had time to update to chromium 124.

Sparticuz avatar Apr 30 '24 16:04 Sparticuz

This issue hit us today but we've not updated for a while so it seems to effect old versions too. Is there anyway to make this project dump a stacktrace when the browser crashes?

You can add puppeteer.launch({dumpio:true}) in the object.

Sparticuz avatar Apr 30 '24 16:04 Sparticuz

We have also started facing same issue. We have not update package version from 3 months. While trying to figure out the issue, we found that it is not working only when html contains https URLs as we had some https URLs in img tag and link tag. With http urls, it is working fine.

We had the following package.json

 "@sparticuz/chromium": "^109.0.5",
 "puppeteer-core": "^19.4.0",

Same thing is working fine with https URLs after updating package version to (as mentioned by @fire015 )

"@sparticuz/chromium": "122.0.0",
"puppeteer-core": "22.5.0"

shubhamvyapar avatar Apr 30 '24 18:04 shubhamvyapar

Having the same issue, not working in our production app.

ryan-concora avatar Apr 30 '24 20:04 ryan-concora

Downgrading to:

    "@sparticuz/chromium": "122.0.0",
    "puppeteer-core": "22.5.0"

Seems to have worked. Leaving this open just to verify I didnt make a mistake selecting the wrong package versions?

Solved.

In my case, Downgrading to version as you describe and additionally I have to install "web-streams-polyfill" package. And added this line : global.ReadableStream = require('web-streams-polyfill').ReadableStream;

urvik-bhalala avatar Apr 30 '24 21:04 urvik-bhalala

We've worked around it by rolling back to the previous Lambda Runtime ARN.

leepa avatar Apr 30 '24 21:04 leepa

We've worked around it by rolling back to the previous Lambda Runtime ARN.

How do you do this with Serverless framework?

ryan-concora avatar Apr 30 '24 22:04 ryan-concora

We also ran into this today, the version changes suggested above:

"@sparticuz/chromium": "122.0.0",
"puppeteer-core": "22.5.0"

and in our case an upgrade of the Chromium Lambda Layer to latest was the change combination that got us back up and running.

iLykaPaul avatar Apr 30 '24 22:04 iLykaPaul

Ours doesn't seem to want to pull from s3

"An executablePathorchannelmust be specified forpuppeteer-core" name:"Error" stacktrace:"at assert (/var/task/node_modules/puppeteer-core/lib/cjs/puppeteer/util/assert.js:18:15) at ChromeLauncher.computeLaunchArguments (/var/task/node_modules/puppeteer-core/lib/cjs/puppeteer/node/ChromeLauncher.js:102:36) at async ChromeLauncher.launch (/var/task/node_modules/puppeteer-core/lib/cjs/puppeteer/node/ProductLauncher.js:70:28) at async CoverPageGenerator.generatePDF (/var/task/src/CoverPage.js:60:19) at async _getCoverPage (/var/task/src/zip.js:72:10) at async exports.createSubmittal (/var/task/src/zip.js:33:21) at async Worker.work (/var/task/src/worker.js:25:27) at async post (/var/task/src/handler.js:19:3)" type:"ERROR_TYPE_UNCAUGHT"

ryan-concora avatar Apr 30 '24 22:04 ryan-concora

We've worked around it by rolling back to the previous Lambda Runtime ARN.

How do you do this with Serverless framework?

@ryan-concora Specify the run time management at the "provider" or "function" level in your serverless.yml file:

functions:
  hello:
    runtimeManagement:
      mode: manual
      arn: arn:aws:lambda:us-east-1::runtime:0cdcfbdefbc5e7d3343f73c2e2dd3cba17d61dea0686b404502a0c9ce83931b9

The ARN version in the example above works for me (The offending ARN is mentioned in OP: https://github.com/Sparticuz/chromium/issues/271#issue-2271372222) Reference: https://www.serverless.com/framework/docs-providers-aws-guide-functions

xanharris avatar May 01 '24 02:05 xanharris

Facing the same issue.

Error: spawn Unknown system error -8

"@sparticuz/chromium": "^110.0.0",
"puppeteer": "19.6.0"
const browser = await puppeteer.launch({
    dumpio: true,
    args: chromium.args,
    executablePath: await chromium.executablePath(),
    headless: true,
    ignoreHTTPSErrors: true,
})

Tried upgrading to

Downgrading to:

    "@sparticuz/chromium": "122.0.0",
    "puppeteer-core": "22.5.0"

Seems to have worked. Leaving this open just to verify I didnt make a mistake selecting the wrong package versions?

This still did not work. Any other ideas ??

gaurav112 avatar May 01 '24 08:05 gaurav112

Facing the same issue.

Error: spawn Unknown system error -8

"@sparticuz/chromium": "^110.0.0",
"puppeteer": "19.6.0"
const browser = await puppeteer.launch({
    dumpio: true,
    args: chromium.args,
    executablePath: await chromium.executablePath(),
    headless: true,
    ignoreHTTPSErrors: true,
})

Tried upgrading to

Downgrading to:

    "@sparticuz/chromium": "122.0.0",
    "puppeteer-core": "22.5.0"

Seems to have worked. Leaving this open just to verify I didnt make a mistake selecting the wrong package versions?

This still did not work. Any other ideas ??

Our solution:

  • Added lambda layer, pulling from s3 doesn't seem to be working
  • Used this as executable path (examples need updating chromium.executablePath('/opt/nodejs/node_modules/@sparticuz/chromium/bin';
  • Removed "@sparticuz/chromium-min": "122.0.0"
  • Added as dev dep "@sparticuz/chromium": "122.0.0"
  • "puppeteer-core": "22.5.0"
  • Used v122.0.0 lambda layer

ryan-concora avatar May 01 '24 08:05 ryan-concora

Removing our disabling of the graphics mode solved this for us:

- chromium.setGraphicsMode = false
+ // FIXME: re-enable in Chromium 124
+ // ref: https://github.com/Sparticuz/chromium/issues/271#issuecomment-2085784736
+ // chromium.setGraphicsMode = false

with the following versions:

    "@sparticuz/chromium": "123.0.1",
    "playwright-core": "1.43.1",

edit: actually, not sure about this, we're still seeing some errors

LinusU avatar May 01 '24 08:05 LinusU

We've worked around it by rolling back to the previous Lambda Runtime ARN.

How do you do this with Serverless framework?

change this in the Runtime management configuration:

Screenshot 2024-05-01 at 16 07 42

2803media avatar May 01 '24 14:05 2803media

FWIW I was able to get it working with:

"@sparticuz/chromium": "122.0.0",
"puppeteer-core": "^21.1.1",
"puppeteer-extra": "^3.3.6",
"puppeteer-extra-plugin-stealth": "^2.11.2",

JakeDluhy avatar May 01 '24 15:05 JakeDluhy

setting this works chromium.setGraphicsMode = true;

adefemi avatar May 01 '24 16:05 adefemi

  1. @Sparticuz Little curious to understand why the below version of packages are getting into an error when the latest package version was released? What is the breaking change we have? "@sparticuz/chromium": "^106.0.2" "puppeteer-core": "^18.0.5"
  2. @shubhamvyapar For the suggested downgrade version "@sparticuz/chromium": "122.0.0", "puppeteer-core": "22.5.0", looks like we need Node 18, Can you kindly suggest any compatible version for Node 16 runtime env?

We got into an error saying Navigation failed because browser has disconnected! seems like chrome crashed. Is this the same issue?

AkbarHabeeb avatar May 01 '24 17:05 AkbarHabeeb

setting this works chromium.setGraphicsMode = true;

This worked also for me chromium.setGraphicsMode = true;

puppeteer-core 22.6.0 @sparticuz/chromium 123.0.1,

jmartinezgdev avatar May 01 '24 22:05 jmartinezgdev

I am curious about the root cause, seems it relate to SSL lib update, but why GraphicsMode can make it solved.

casyalex avatar May 02 '24 08:05 casyalex

We got into an error saying Navigation failed because browser has disconnected! seems like chrome crashed. Is this the same issue?

Yes we were getting the same error "Navigation failed because browser has disconnected!", but it was happening in those cases only where were are loading some asset using https, http and setOfflineMode(true) works fine.

vishwassingh47 avatar May 02 '24 08:05 vishwassingh47

This works for us.

We migrated to a previous version of Runtime on lambda(Node.Js 18.*)

ARN: arn:aws:lambda:ap-south-1::runtime:0cdcfbdefbc5e7d3343f73c2e2dd3cba17d61dea0686b404502a0c9ce83931b9

package.json for compatibility.

"@sparticuz/chromium": "^119.0.0",
"puppeteer": "^21.5.2"

gaurav112 avatar May 02 '24 08:05 gaurav112

I am having a similar issue to some of the more recently posted comments in this thread:

Navigation failed because browser has disconnected...

Others are saying that downgrading versions helped them, but I am starting to wonder if there is something else at play here. This has crept up for many of us on all different versions.

Is there a corrupted browser object being cached between lambda executions and deploys?

Is it the downgraded version or the deploy of the downgraded version that is actually fixing the issue? I am wondering if this issue creeps back up for those who have said it has been fixed with a version downgrade.

We started seeing this with these versions, which were previously working for a year. Then all of a sudden we started getting the error.

"@sparticuz/chromium": "114.0.0",
"puppeteer-core": "20.7.3"

Did AWS change something with lambdas and lambda layers that is biting all of us?

bjoepfeiffer avatar May 02 '24 16:05 bjoepfeiffer

Did AWS change something with lambdas and lambda layers that is biting all of us?

Yes, AWS updated some libraries that are causing issues with the libraries included in this package. I'm in contact with the team at AWS and we will be working on a solution.

I would recommend pinning the lambda execution environment while this issue gets sorted.

Sparticuz avatar May 02 '24 16:05 Sparticuz

I'm having a similar issue; it suddenly stopped working and started giving us this error code:

The input directory "/opt/nodejs/node_modules/@sparticuz/chromium/bin/" does not exist. I've tried all the solutions mentioned above, but none of them worked.

tomasalvarez13 avatar May 02 '24 16:05 tomasalvarez13

I rolled back manualy the runtime version and it worked

tomasalvarez13 avatar May 02 '24 16:05 tomasalvarez13

on one of my projects it works, on a different one, same settings same packages it does not work

raducretu avatar May 02 '24 17:05 raducretu

@tomasalvarez13 which runtime version worked for you, specifically?

tyanko1 avatar May 02 '24 17:05 tyanko1

@tomasalvarez13 which runtime version worked for you, specifically?

for me the one that worked on one place and didn't work on a different project is: node 18.x "puppeteer-core": "^22.6.2", chromium-v123.0.1-layer.zip

same configuration on one account works on the other one it does not work, it goes timeout on: const page = await browser.newPage();

raducretu avatar May 02 '24 17:05 raducretu