lost-pixel
lost-pixel copied to clipboard
lost-pixel
Intermittent `ENOENT`
Bug description
During GHA run it couldn't find diff for one of the pages for some reason
How to reproduce
Intermittent, saw it once so far
Expected behavior
No response
Lost Pixel information
lost-pixel logs from CI
Run lost-pixel/[email protected]
/usr/bin/docker run --name lostpixellostpixelv382_0c3f5f --label 09[7](https://github.com/Maxim-Mazurok/maxim.mazurok.com/actions/runs/7328462883/job/19956775494#step:11:8)6f5 --workdir /github/workspace --rm -e "COMPOSER_PROCESS_TIMEOUT" -e "COMPOSER_NO_INTERACTION" -e "COMPOSER_NO_AUDIT" -e "INPUT_FINALIZE" -e "INPUT_CACHE_KEY" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_REPOSITORY_OWNER_ID" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_REPOSITORY_ID" -e "GITHUB_ACTOR_ID" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKFLOW_REF" -e "GITHUB_WORKFLOW_SHA" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_ENVIRONMENT" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e "ACTIONS_RESULTS_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/maxim.mazurok.com/maxim.mazurok.com":"/github/workspace" lostpixel/lost-pixel:v3.[8](https://github.com/Maxim-Mazurok/maxim.mazurok.com/actions/runs/7328462883/job/19956775494#step:11:9).2
Environment:
WORKSPACE=/github/workspace
CI_BUILD_ID=7328462883
CI_BUILD_NUMBER=61
EVENT_PATH=/github/workflow/event.json
COMMIT_HASH=b378d5db8ca5ba04a4ee[9](https://github.com/Maxim-Mazurok/maxim.mazurok.com/actions/runs/7328462883/job/19956775494#step:11:10)9d8bc4141de877[10](https://github.com/Maxim-Mazurok/maxim.mazurok.com/actions/runs/7328462883/job/19956775494#step:11:11)8f6
COMMIT_SHA=b378d5db8ca5ba04a4ee99d8bc4[14](https://github.com/Maxim-Mazurok/maxim.mazurok.com/actions/runs/7328462883/job/19956775494#step:11:15)1de877108f6
COMMIT_REF_NAME=main
REPOSITORY=Maxim-Mazurok/maxim.mazurok.com
Version: 3.8.2
Loading project config ...
Current working directory: /github/workspace
Looking for config file: /github/workspace/lostpixel.config.(ts|js|cjs|mjs)
✅ Found config file: /github/workspace/lostpixel.config.ts
🚀 Starting Lost Pixel in 'generateOnly' mode
📂 Creating shot folders
📸 Creating shots
Removing 0 files from .lostpixel/current/
Removing 0 files from .lostpixel/difference/
Found 0 pages from external loader
=== [Page Mode] http://[17](https://github.com/Maxim-Mazurok/maxim.mazurok.com/actions/runs/7328462883/job/19956775494#step:11:18)2.17.0.1:8080 ===
Prepared 8 pages for screenshots
[1/8] Taking screenshot of '/!index ' (/!index)
[2/8] Taking screenshot of '/blog/!index ' (/blog/!index)
[3/8] Taking screenshot of '/blog/socials-calendar ' (/blog/socials-calendar)
[4/8] Taking screenshot of '/blog/typescript-excess-property-checks ' (/blog/typescript-excess-property-checks)
[5/8] Taking screenshot of '/blog/google-ads-custom-conversion-setup ' (/blog/google-ads-custom-conversion-setup)
[1/8] Screenshot of '/!index' taken and saved to '.lostpixel/current/!index.png' in 3.072s (/!index)
[6/8] Taking screenshot of '/blog/ignore-internal-traffic-in-google-analytics ' (/blog/ignore-internal-traffic-in-google-analytics)
[3/8] Screenshot of '/blog/socials-calendar' taken and saved to '.lostpixel/current/blog/socials-calendar.png' in 3.092s (/blog/socials-calendar)
[7/8] Taking screenshot of '/blog/visual-testing-percy-gha-slack ' (/blog/visual-testing-percy-gha-slack)
[5/8] Screenshot of '/blog/google-ads-custom-conversion-setup' taken and saved to '.lostpixel/current/blog/google-ads-custom-conversion-setup.png' in 3.570s (/blog/google-ads-custom-conversion-setup)
[8/8] Taking screenshot of '/blog/meta-pixel-traps-in-app-browsers ' (/blog/meta-pixel-traps-in-app-browsers)
[2/8] Screenshot of '/blog/!index' taken and saved to '.lostpixel/current/blog/!index.png' in 4.341s (/blog/!index)
[4/8] Screenshot of '/blog/typescript-excess-property-checks' taken and saved to '.lostpixel/current/blog/typescript-excess-property-checks.png' in 4.347s (/blog/typescript-excess-property-checks)
[7/8] Screenshot of '/blog/visual-testing-percy-gha-slack' taken and saved to '.lostpixel/current/blog/visual-testing-percy-gha-slack.png' in 4.6[26](https://github.com/Maxim-Mazurok/maxim.mazurok.com/actions/runs/7328462883/job/19956775494#step:11:27)s (/blog/visual-testing-percy-gha-slack)
[6/8] Screenshot of '/blog/ignore-internal-traffic-in-google-analytics' taken and saved to '.lostpixel/current/blog/ignore-internal-traffic-in-google-analytics.png' in 4.689s (/blog/ignore-internal-traffic-in-google-analytics)
[8/8] Screenshot of '/blog/meta-pixel-traps-in-app-browsers' taken and saved to '.lostpixel/current/blog/meta-pixel-traps-in-app-browsers.png' in 5.[34](https://github.com/Maxim-Mazurok/maxim.mazurok.com/actions/runs/7328462883/job/19956775494#step:11:35)7s (/blog/meta-pixel-traps-in-app-browsers)
Screenshots done!
Creating shots took 9.174 seconds
🔍 Checking differences
Comparing 8 screenshots using 'pixelmatch' as compare engine
[1/8] Comparing '/!index' (/!index)
[2/8] Comparing '/blog/!index' (/blog/!index)
[3/8] Comparing '/blog/socials-calendar' (/blog/socials-calendar)
[4/8] Comparing '/blog/typescript-excess-property-checks' (/blog/typescript-excess-property-checks)
[5/8] Comparing '/blog/google-ads-custom-conversion-setup' (/blog/google-ads-custom-conversion-setup)
[6/8] Comparing '/blog/ignore-internal-traffic-in-google-analytics' (/blog/ignore-internal-traffic-in-google-analytics)
[7/8] Comparing '/blog/visual-testing-percy-gha-slack' (/blog/visual-testing-percy-gha-slack)
[8/8] Comparing '/blog/meta-pixel-traps-in-app-browsers' (/blog/meta-pixel-traps-in-app-browsers)
[1/8] No difference found. (/!index)
[2/8] No difference found. (/blog/!index)
[3/8] No difference found. (/blog/socials-calendar)
[4/8] No difference found. (/blog/typescript-excess-property-checks)
[5/8] No difference found. (/blog/google-ads-custom-conversion-setup)
[6/8] No difference found. (/blog/ignore-internal-traffic-in-google-analytics)
[7/8] No difference found. (/blog/visual-testing-percy-gha-slack)
❌ ENOENT: no such file or directory, open '.lostpixel/difference/blog/meta-pixel-traps-in-app-browsers.png'
Sending anonymized telemetry data.
hey @Maxim-Mazurok! thanks for the reporting! This usually happens for reasons not related to lost-pixel execution, when our runner could not open a page, for example, or the page errored by itself. My usual recommendation is to first make sure that the implementation is ok and that nothing breaks when you run your app on CI, and then, by exclusion method, try to figure out what is wrong.
If it gets reproducible though - would be super happy to look into that with you and help with the fix!
Got the same when running locally:
=== [Page Mode] http://localhost:8080/ ===
Prepared 8 pages for screenshots
[1/8] Taking screenshot of '/!index ' (/!index)
[2/8] Taking screenshot of '/blog/!index ' (/blog/!index)
[3/8] Taking screenshot of '/blog/socials-calendar ' (/blog/socials-calendar)
[4/8] Taking screenshot of '/blog/typescript-excess-property-checks ' (/blog/typescript-excess-property-checks)
[5/8] Taking screenshot of '/blog/google-ads-custom-conversion-setup ' (/blog/google-ads-custom-conversion-setup)
[2/8] Screenshot of '/blog/!index' taken and saved to '.lostpixel/current/blog/!index.png' in 2.250s (/blog/!index)
[6/8] Taking screenshot of '/blog/ignore-internal-traffic-in-google-analytics ' (/blog/ignore-internal-traffic-in-google-analytics)
[3/8] Screenshot of '/blog/socials-calendar' taken and saved to '.lostpixel/current/blog/socials-calendar.png' in 2.254s (/blog/socials-calendar)
[7/8] Taking screenshot of '/blog/visual-testing-percy-gha-slack ' (/blog/visual-testing-percy-gha-slack)
[4/8] Screenshot of '/blog/typescript-excess-property-checks' taken and saved to '.lostpixel/current/blog/typescript-excess-property-checks.png' in 2.585s (/blog/typescript-excess-property-checks)
[8/8] Taking screenshot of '/blog/meta-pixel-traps-in-app-browsers ' (/blog/meta-pixel-traps-in-app-browsers)
[1/8] Screenshot of '/!index' taken and saved to '.lostpixel/current/!index.png' in 2.881s (/!index)
[5/8] Screenshot of '/blog/google-ads-custom-conversion-setup' taken and saved to '.lostpixel/current/blog/google-ads-custom-conversion-setup.png' in 2.884s (/blog/google-ads-custom-conversion-setup)
[7/8] Screenshot of '/blog/visual-testing-percy-gha-slack' taken and saved to '.lostpixel/current/blog/visual-testing-percy-gha-slack.png' in 3.012s (/blog/visual-testing-percy-gha-slack)
[6/8] Screenshot of '/blog/ignore-internal-traffic-in-google-analytics' taken and saved to '.lostpixel/current/blog/ignore-internal-traffic-in-google-analytics.png' in 3.020s (/blog/ignore-internal-traffic-in-google-analytics)
[8/8] Screenshot of '/blog/meta-pixel-traps-in-app-browsers' taken and saved to '.lostpixel/current/blog/meta-pixel-traps-in-app-browsers.png' in 3.336s (/blog/meta-pixel-traps-in-app-browsers)
Screenshots done!
Creating shots took 6.138 seconds
🔍 Checking differences
Comparing 8 screenshots using 'pixelmatch' as compare engine
[1/8] Comparing '/!index' (/!index)
[2/8] Comparing '/blog/!index' (/blog/!index)
[3/8] Comparing '/blog/socials-calendar' (/blog/socials-calendar)
[4/8] Comparing '/blog/typescript-excess-property-checks' (/blog/typescript-excess-property-checks)
[5/8] Comparing '/blog/google-ads-custom-conversion-setup' (/blog/google-ads-custom-conversion-setup)
[6/8] Comparing '/blog/ignore-internal-traffic-in-google-analytics' (/blog/ignore-internal-traffic-in-google-analytics)
[7/8] Comparing '/blog/visual-testing-percy-gha-slack' (/blog/visual-testing-percy-gha-slack)
[8/8] Comparing '/blog/meta-pixel-traps-in-app-browsers' (/blog/meta-pixel-traps-in-app-browsers)
[1/8] Difference of 126891 pixels (3.99%) found. Difference image saved to: .lostpixel/difference/!index.png (/!index)
❌ ENOENT: no such file or directory, open '.lostpixel/difference/blog/!index.png'
Just to clarify on the above message - ENOENT: no such file or directory, open means lost-pixel did not make the screenshot of this page, means lost-pixel could not open this page properly in playwright. So first things first we need to make sure that the page you are trying to open exists & you can open it in the browser.
if you can repro this locally - awesome, please verify that /!index page is accessible by the browser 🙌
Just to clarify on the above message -
ENOENT: no such file or directory, openmeans lost-pixel did not make the screenshot of this page, means lost-pixel could not open this page properly in playwright. So first things first we need to make sure that the page you are trying to open exists & you can open it in the browser.if you can repro this locally - awesome, please verify that
/!indexpage is accessible by the browser 🙌
Hmm, interesting, but I don't see errors regarding screenshot taking... And on local I do see the screenshot in current folder. It's the difference folder that is missing a file. I would assume it's some sort of a race
that is interesting indeed! is it always this !index page or?
Is there anything special about the page?
can you run successfully locally at least once, or it fails everytime now?
On local it is the same blog/!index page and I can't get it working at all now, fails every time
On GHA it was another page - ❌ ENOENT: no such file or directory, open '.lostpixel/difference/blog/meta-pixel-traps-in-app-browsers.png'
Nothing special about the pages I think... They all are here: https://maxim.mazurok.com/
I might try remove ! from the name
Looks like it's related to / in the names.
I did this and it works fine now:
import { CustomProjectConfig } from 'lost-pixel';
import { getUrlsFromSitemap } from './src/helpers';
import { readFileSync } from 'fs';
export const config: CustomProjectConfig = {
pageShots: {
pages: getUrlsFromSitemap(readFileSync('./sitemap.xml', 'utf-8')).map(
x => ({
path: x.pathname,
name: (x.pathname.endsWith('/') ? `${x.pathname}index` : x.pathname)
.replace(/\//g, '_')
.replace(/^_/g, ''),
})
),
baseUrl: 'http://172.17.0.1:8080', // this has to be internal IP of the docker container inside of Github Actions
},
generateOnly: true,
failOnDifference: true,
};
Prepared 8 pages for screenshots
[1/8] Taking screenshot of 'index ' (index)
[2/8] Taking screenshot of 'blog_index ' (blog_index)
[3/8] Taking screenshot of 'blog_socials-calendar ' (blog_socials-calendar)
[4/8] Taking screenshot of 'blog_typescript-excess-property-checks ' (blog_typescript-excess-property-checks)
[5/8] Taking screenshot of 'blog_google-ads-custom-conversion-setup ' (blog_google-ads-custom-conversion-setup)
[2/8] Screenshot of 'blog_index' taken and saved to '.lostpixel/current/blog_index.png' in 2.259s (blog_index)
[6/8] Taking screenshot of 'blog_ignore-internal-traffic-in-google-analytics ' (blog_ignore-internal-traffic-in-google-analytics)
[3/8] Screenshot of 'blog_socials-calendar' taken and saved to '.lostpixel/current/blog_socials-calendar.png' in 2.262s (blog_socials-calendar)
[7/8] Taking screenshot of 'blog_visual-testing-percy-gha-slack ' (blog_visual-testing-percy-gha-slack)
[1/8] Screenshot of 'index' taken and saved to '.lostpixel/current/index.png' in 2.829s (index)
[8/8] Taking screenshot of 'blog_meta-pixel-traps-in-app-browsers ' (blog_meta-pixel-traps-in-app-browsers)
[4/8] Screenshot of 'blog_typescript-excess-property-checks' taken and saved to '.lostpixel/current/blog_typescript-excess-property-checks.png' in 2.830s (blog_typescript-excess-property-checks)
[5/8] Screenshot of 'blog_google-ads-custom-conversion-setup' taken and saved to '.lostpixel/current/blog_google-ads-custom-conversion-setup.png' in 2.833s (blog_google-ads-custom-conversion-setup)
[6/8] Screenshot of 'blog_ignore-internal-traffic-in-google-analytics' taken and saved to '.lostpixel/current/blog_ignore-internal-traffic-in-google-analytics.png' in 3.884s (blog_ignore-internal-traffic-in-google-analytics)
[8/8] Screenshot of 'blog_meta-pixel-traps-in-app-browsers' taken and saved to '.lostpixel/current/blog_meta-pixel-traps-in-app-browsers.png' in 3.318s (blog_meta-pixel-traps-in-app-browsers)
[7/8] Screenshot of 'blog_visual-testing-percy-gha-slack' taken and saved to '.lostpixel/current/blog_visual-testing-percy-gha-slack.png' in 3.887s (blog_visual-testing-percy-gha-slack)
Screenshots done!
Creating shots took 6.385 seconds
🔍 Checking differences
Comparing 8 screenshots using 'pixelmatch' as compare engine
[1/8] Comparing 'index' (index)
[2/8] Comparing 'blog_index' (blog_index)
[3/8] Comparing 'blog_socials-calendar' (blog_socials-calendar)
[4/8] Comparing 'blog_typescript-excess-property-checks' (blog_typescript-excess-property-checks)
[5/8] Comparing 'blog_google-ads-custom-conversion-setup' (blog_google-ads-custom-conversion-setup)
[6/8] Comparing 'blog_ignore-internal-traffic-in-google-analytics' (blog_ignore-internal-traffic-in-google-analytics)
[7/8] Comparing 'blog_visual-testing-percy-gha-slack' (blog_visual-testing-percy-gha-slack)
[8/8] Comparing 'blog_meta-pixel-traps-in-app-browsers' (blog_meta-pixel-traps-in-app-browsers)
[1/8] Difference of 126891 pixels (3.99%) found. Difference image saved to: .lostpixel/difference/index.png (index)
[2/8] Difference of 19880 pixels (2.16%) found. Difference image saved to: .lostpixel/difference/blog_index.png (blog_index)
[3/8] Difference of 30770 pixels (2.02%) found. Difference image saved to: .lostpixel/difference/blog_socials-calendar.png (blog_socials-calendar)
[4/8] Difference of 1533451 pixels (18.73%) found. Difference image saved to: .lostpixel/difference/blog_typescript-excess-property-checks.png (blog_typescript-excess-property-checks)
[5/8] Difference of 908848 pixels (14.71%) found. Difference image saved to: .lostpixel/difference/blog_google-ads-custom-conversion-setup.png (blog_google-ads-custom-conversion-setup)
[6/8] Difference of 814749 pixels (13.27%) found. Difference image saved to: .lostpixel/difference/blog_ignore-internal-traffic-in-google-analytics.png (blog_ignore-internal-traffic-in-google-analytics)
[7/8] Difference of 2182673 pixels (20.39%) found. Difference image saved to: .lostpixel/difference/blog_visual-testing-percy-gha-slack.png (blog_visual-testing-percy-gha-slack)
[8/8] Difference of 2435290 pixels (14.71%) found. Difference image saved to: .lostpixel/difference/blog_meta-pixel-traps-in-app-browsers.png (blog_meta-pixel-traps-in-app-browsers)
Comparison done!
👋 Exiting process with 8 found differences & 0 baselines to update
Sending anonymized telemetry data.
hmm, great catch! wanna give a proper fix a spin? :D I think you are already pretty much caught the exact problem so fix might not be that complicated 🤔
Well, the exact problem is that lost-pixel doesn't work great with folders... So we could either a - replace all / in names with _ or something. Or b - make lost-pixel work well with folders.
I think option b is the best, but I don't have bandwidth for now. I am just annoyed that percy is giving me 1px diff every day, so trying out lost-pixel, if it works well for me I might consider adding folders support, but tbh probably not anytime soon as I don't have many pages for now. Anyway, hope this helps, cheers!
yup, some legit good thoughts, thanks Maxim! we are happy to help you with your migration from Percy anytime both for personal & work projects 😉 we have bunch of people migrating & the fact that you could choose individual thresholds on per page level is usually helping a ton already with those 1-2px flakiness