webperf_core icon indicating copy to clipboard operation
webperf_core copied to clipboard

Missing a11y statement gives 5.0 rating

Open krompaco opened this issue 5 months ago • 7 comments

URL

https://krompaco.nu

Webperf_core version

2025.6.5

What environment are you running?

  • [ ] Using GitHub Actions
  • [x] Using Docker Image from Dock Hub
  • [ ] Local machine
  • [ ] Other, please describe your environment in How to reproduce.

Has anyone else reported it already?

  • [x] No, there are no other issues related to this in Issues.

Expected Behavior

I assume 1.0 would be correct?

Actual Behavior.

Rating is 5.0

How to reproduce

python3 default.py -u https://krompaco.nu -t 26

Log output

### Rating:
- Overall: 5.0

### Review:

#### Overall:

Cleaning cache files...

krompaco avatar Jul 01 '25 10:07 krompaco

I'm running 2025.6.5 on a lot of sites now... Seems like maybe Lighthouse -t 30 also has this behavior, getting 5.0 if something crashes or is entirely unprocessed. I'll create isolated tickets per test when I have more info.

krompaco avatar Jul 01 '25 11:07 krompaco

Testing website https://krompaco.nu

Started: 2025-07-01 11:55:29 Finished: 2025-07-01 11:55:49

Rating:

  • Overall: 5.0

Review:

Overall:

Data:

{
  "version": "2025.6.0",
  "dependencies": {
     "@sitespeed.io/plugin": "1.0.0",
     "@sitespeed.io/log": "0.2.6"
  },
  "groups": {
     "krompaco.nu": {
        "version": "2025.6.0",
        "dependencies": {
           "@sitespeed.io/plugin": "1.0.0",
           "@sitespeed.io/log": "0.2.6"
        },
        "score": {
           "overall": 100
        },
        "issues": [],
        "pages": {
           "https://krompaco.nu": {
              "version": "2025.6.0",
              "dependencies": {
                 "@sitespeed.io/plugin": "1.0.0",
                 "@sitespeed.io/log": "0.2.6"
              },
              "url": "https://krompaco.nu",
              "score": {
                 "overall": 100
              },
              "issues": []
           }
        }
     }
  }
}

cockroacher avatar Jul 01 '25 12:07 cockroacher

We should solve below issue while looking at current issue.

https://github.com/Webperf-se/plugin-accessibility-statement/issues/48

cockroacher avatar Jul 01 '25 12:07 cockroacher

I think I noticed that 404 test also sets 5.0 when it can't process content.

krompaco avatar Jul 02 '25 07:07 krompaco

When i run the plugin-accessibility-statement directly in sitespeed for your url i get the below. So it seem to tell webperf-core there are critical errors.

{
   "start-url": "https://krompaco.nu/",
   "analyzedData": [
      {
         "url": "https://krompaco.nu/",
         "htmls": [
            {
               "url": "https://krompaco.nu/",
               "content": "<!DOCTYPE html>\n<html lang=\"en\"><head><meta charset=\"utf-8\">\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">\n    <title>Blog posts</title>\n    <!-- Webfonts: Copyright © Letters From Sweden. -->\n    <link href=\"/dist/funkis-abc-variable.woff2\" rel=\"preload\" as=\"font\" type=\"font/woff2\" crossorigin=\"anonymous\">\n    <link href=\"/dist/styles.css?v=da593e819af1467d047173f7f0b19e95\" rel=\"stylesheet\">\n    <link rel=\"shortcut icon\" href=\"/favicon.ico\" type=\"image/ico\"><meta name=\"description\" content=\"This is a blog where I post computer stuff information I&#x27;ve found useful.\"><meta name=\"generator\" content=\"Record Collector\">\n    <link rel=\"alternate\" type=\"application/rss&#x2B;xml\" title=\"Blog posts\" href=\"[https://krompaco.nu/rss.xml\](https://krompaco.nu/rss.xml/)"><link rel=\"canonical\" href=\"[https://krompaco.nu/\](https://krompaco.nu//)"></head>\n<body class=\"bg-white text-gray-700 font-default-weight\"><header class=\"bg-white\"><div class=\"absolute inset
               "index": 1
            }
         ]
      }
   ],
   "interesting-urls": {},
   "knowledgeData": [
      {
         "url": "https://krompaco.nu/",
         "group": "krompaco.nu",
         "issues": {
            "no-a11y-statement": {
               "test": "a11y-statement",
               "rule": "no-a11y-statement",
               "category": "a11y",
               "text": "Unable to find accessibility statement",
               "severity": "critical",
               "subIssues": [
                  {
                     "rule": "no-a11y-statement",
                     "category": "a11y",
                     "text": "Unable to find accessibility statement",
                     "severity": "critical",
                     "line": 0,
                     "column": 0
                  }
               ]
            }
         },
         "interesting-urls": {},
         "body-text": "Blog posts Upgraded a CMS 11 solution that was using EPiFocalPoint. Published 2025-06-02 More early findings from using a SaaS CMS instance; setting up Graph queries that works for both visitor pageviews and editor previews. Published 2025-04-14 Introducing a tool I put together for exploring an Optimizely SaaS CMS instance and to source code control definitions. Published 2025-03-11 Been spending some time in a SaaS CMS instance trying to learn how to best use the new Visual Builder grids and how to control the whole content type structure from code. Published 2025-02-07 A recommendation when on DXP, or in some other ASP.NET app behind Cloudflare, to get a more exact geo-lookup logged. Published 2024-06-10 Taking a look at how to run your own imgproxy instance on Azure. Published 2024-05-05 Needed to synchronize my OMVP Webperf Leaderboard with the current OMVP list and decided to automate it a little bit. Published 2024-01-03 An editor needs to regularly put static site output in thei
      }
   ],
   "visited-urls": {}
}

7h3Rabbit avatar Jul 05 '25 22:07 7h3Rabbit

To be compared with for example eskilstuna.se:

{
   "start-url": "https://eskilstuna.se/",
   "analyzedData": [
      {
         "url": "https://eskilstuna.se/",
         "htmls": [
            {
               "url": "https://www.eskilstuna.se/",
               "content": "<!DOCTYPE html>\n<html lang=\"sv\" class=\"sv-no-js sv-template-startsida\">\n<head>\n   <meta charset=\"UTF-8\">\n   <script nonce=\"aaa3a290-59ec-11f0-8176-d55b9fbcb50b\">(function(H){H.className=H.className.replace(/\\bsv-no-js\\b/,'sv-js')})(document.documentElement)</script>\n   <title>Eskilstuna kommun</title>\n   <link rel=\"preload\" href=\"/sitevision/system-resource/a1b2d0ea0bca8d1faecf78e212ff7c9459449bdfe0b1362ba2d18131dd70884f/js/jquery-slim.js\" as=\"script\">\n   <link rel=\"preload\" href=\"/sitevision/system-resource/a1b2d0ea0bca8d1faecf78e212ff7c9459449bdfe0b1362ba2d18131dd70884f/envision/envision.js\" as=\"script\">\n   <link rel=\"preload\" href=\"/sitevision/system-resource/a1b2d0ea0bca8d1faecf78e212ff7c9459449bdfe0b1362ba2d18131dd70884f/js/utils-slim.js\" as=\"script\">\n   <link rel=\"preload\" href=\"/sitevision/system-resource/a1b2d0ea0bca8d1faecf78e212ff7c9459449bdfe0b1362ba2d18131dd70884f/js/portlets-slim.js\" as=\"script\">\n   <meta name=\"viewport\" con
               "index": 1
            }
         ]
      },
      {
         "url": "https://eskilstuna.se/ovrigt/om-webbplatsen/tillganglighetsredogorelser",
         "htmls": [
            {
               "url": "https://www.eskilstuna.se/ovrigt/om-webbplatsen/tillganglighetsredogorelser",
               "content": "<!DOCTYPE html>\n<html lang=\"sv\" class=\"sv-no-js sv-template-innehallssida\">\n<head>\n   <meta charset=\"UTF-8\">\n   <script nonce=\"b37b1fb0-59ec-11f0-8176-d55b9fbcb50b\">(function(H){H.className=H.className.replace(/\\bsv-no-js\\b/,'sv-js')})(document.documentElement)</script>\n   <title>Tillgänglighets­redogörelser - Eskilstuna kommun</title>\n   <link rel=\"preload\" href=\"/sitevision/system-resource/a1b2d0ea0bca8d1faecf78e212ff7c9459449bdfe0b1362ba2d18131dd70884f/js/jquery-slim.js\" as=\"script\">\n   <link rel=\"preload\" href=\"/sitevision/system-resource/a1b2d0ea0bca8d1faecf78e212ff7c9459449bdfe0b1362ba2d18131dd70884f/envision/envision.js\" as=\"script\">\n   <link rel=\"preload\" href=\"/sitevision/system-resource/a1b2d0ea0bca8d1faecf78e212ff7c9459449bdfe0b1362ba2d18131dd70884f/js/utils-slim.js\" as=\"script\">\n   <link rel=\"preload\" href=\"/sitevision/system-resource/a1b2d0ea0bca8d1faecf78e212ff7c9459449bdfe0b1362ba2d18131dd70884f/js/portlets-slim.js\" as=\"script
               "index": 1
            }
         ]
      }
   ],
   "interesting-urls": {
      "https://eskilstuna.se/ovrigt/om-webbplatsen/tillganglighetsredogorelser": 0.5,
      "https://eskilstuna.se/tillganglighetsguiden": 0.35,
      "https://eskilstuna.se/ovrigt/om-webbplatsen": 0.29,
      "https://eskilstuna.se/ovrigt/om-webbplatsen/om-kakor": 0.25
   },
   "knowledgeData": [
      {
         "url": "https://eskilstuna.se/",
         "group": "eskilstuna.se",
         "issues": {},
         "interesting-urls": {
            "https://eskilstuna.se/ovrigt/om-webbplatsen/tillganglighetsredogorelser": 0.5,
            "https://eskilstuna.se/tillganglighetsguiden": 0.35,
            "https://eskilstuna.se/ovrigt/om-webbplatsen": 0.29,
            "https://eskilstuna.se/ovrigt/om-webbplatsen/om-kakor": 0.25
         },
         "body-text": "Välkommen till eskilstuna.se Populärt E-tjänster Våra webbplatser Aktuellt 30 juni 2025 Efter två intensiva veckor på sommarens lovskola har niondeklassare från Eskilstunas kommunala skolor klarat 34 fler betyg. Dessutom har 57 åttondeklassare stärkt sina kunskaper inför sitt sista år i grundskolan. 19 juni 2025 Den beskriver vad kommunfullmäktige vill uppnå under det kommande året och hur resurserna i budgeten fördelats. 18 juni 2025 Vill du starta, etablera eller utveckla företag i Eskilstuna? Nu lanserar vi en ny webbplats som samlar kommunens stöd och tjänster för företagare – allt på ett ställe. Driftinformation Kontakta Eskilstuna direkt Telefon Sommartid: Måndag-fredag kl. 8.00-16.00 E-post Postadress Eskilstuna kommun631 86 Eskilstuna Genvägar"
      },
      {
         "url": "https://eskilstuna.se/ovrigt/om-webbplatsen/tillganglighetsredogorelser",
         "group": "eskilstuna.se",
         "issues": {
            "compatible-word-partly": {
               "test": "a11y-statement",
               "rule": "compatible-word-partly",
               "category": "a11y",
               "severity": "error",
               "subIssues": [
                  {
                     "url": "https://www.eskilstuna.se/ovrigt/om-webbplatsen/tillganglighetsredogorelser",
                     "rule": "compatible-word-partly",
                     "category": "a11y",
                     "text": "Self-indicates that the website is only partially compliant with the legal requirements",
                     "severity": "error"
                  }
               ]
            },
            "no-network": {
               "test": "a11y-statement",
               "rule": "no-network",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "no-updated-date": {
               "test": "a11y-statement",
               "rule": "no-updated-date",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "updated-date-older-than-1years": {
               "test": "a11y-statement",
               "rule": "updated-date-older-than-1years",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "updated-date-older-than-2years": {
               "test": "a11y-statement",
               "rule": "updated-date-older-than-2years",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "updated-date-older-than-3years": {
               "test": "a11y-statement",
               "rule": "updated-date-older-than-3years",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "updated-date-older-than-4years": {
               "test": "a11y-statement",
               "rule": "updated-date-older-than-4years",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "updated-date-older-than-5years": {
               "test": "a11y-statement",
               "rule": "updated-date-older-than-5years",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "no-evaluation-method": {
               "test": "a11y-statement",
               "rule": "no-evaluation-method",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "has-canonical-notification-function-link": {
               "test": "a11y-statement",
               "rule": "has-canonical-notification-function-link",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "has-old-notification-function-link": {
               "test": "a11y-statement",
               "rule": "has-old-notification-function-link",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "no-notification-function-link": {
               "test": "a11y-statement",
               "rule": "no-notification-function-link",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "compatible-word-not": {
               "test": "a11y-statement",
               "rule": "compatible-word-not",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "no-compatible-word": {
               "test": "a11y-statement",
               "rule": "no-compatible-word",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "has-unreasonably-burdensome-accommodation": {
               "test": "a11y-statement",
               "rule": "has-unreasonably-burdensome-accommodation",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            },
            "no-a11y-statement": {
               "test": "a11y-statement",
               "rule": "no-a11y-statement",
               "category": "a11y",
               "severity": "resolved",
               "subIssues": []
            }
         },
         "interesting-urls": {
            "https://eskilstuna.se/ovrigt/om-webbplatsen/tillganglighetsredogorelser": 0.5,
            "https://eskilstuna.se/tillganglighetsguiden": 0.35,
            "https://eskilstuna.se/ovrigt/om-webbplatsen": 0.29,
            "https://eskilstuna.se/ovrigt/om-webbplatsen/om-kakor": 0.25
         },
         "body-text": "Tillgänglighetsredogörelser Den här sidan beskriver hur Eskilstuna kommun uppfyller lagen om tillgänglighet till digital offentlig service sina webbplatser. Eventuella kända tillgänglighetsproblem redovisas och information om hur du kan rapportera brister till oss finns här. Eskilstuna kommun står bakom de här webbplatserna. Vi vill att så många som möjligt ska kunna använda den. På den här sidan så beskriver vi hur webbplatserna uppfyller lagen om tillgänglighet till digital offentlig service, eventuella kända tillgänglighetsproblem och hur du kan rapportera brister till oss så att vi kan åtgärda dem.Vad kan du göra om du inte kan använda delar av webbplatsen?Om du behöver innehåll från Eskilstuna kommuns webbplatser som inte är tillgängligt för dig, men som är undantaget från lagens tillämpningsområde enligt beskrivning nedan, kan du kontakta oss.Använd kontaktformuläret längst ner på sidan för att meddela oss, skicka e-post eller ringa.Svarstiden är normalt 1 arbetsdag. Lista över ti
         "compatible-word": "delvis förenlig",
         "compatible-text": "Den här webbplatsen är delvis förenlig med lagen om tillgänglighet till digital offentlig service, på grund av de brister som beskrivs nedan.",
         "notification-function-link-text": "Anmäl tillsynsärende hos DIGG",
         "notification-function-link-url": "https://www.digg.se/tdosanmalan",
         "h1": "Tillgänglighetsredogörelser",
         "is-a11y-statement": true,
         "evaluation-method-word": "Hur vi testat webbplats",
         "evaluation-method-text": "Hur vi testat webbplats",
         "dates": [
            {
               "word": "bedömningen",
               "text": "Senaste bedömningen gjordes den 13 januari 2025.",
               "type": "bedömning",
               "date": [
                  2025,
                  1,
                  13
               ],
               "weight": 1
            },
            {
               "word": "bedömningen",
               "text": "Senaste bedömningen gjordes den 17 juni 2025.",
               "type": "bedömning",
               "date": [
                  2025,
                  6,
                  17
               ],
               "weight": 1
            },
            {
               "word": "bedömningen",
               "text": "Senaste bedömningen gjordes den 13 januari 2025.",
               "type": "bedömning",
               "date": [
                  2025,
                  1,
                  1
               ],
               "weight": 1
            },
            {
               "word": "bedömningen",
               "text": "Senaste bedömningen gjordes den 17 juni 2025.",
               "type": "bedömning",
               "date": [
                  2025,
                  6,
                  1
               ],
               "weight": 1
            },
            {
               "word": ".Redogörelsen",
               "text": "Redogörelsen uppdaterades senast den 13 januari 2025",
               "type": "redogörelse",
               "date": [
                  2025,
                  1,
                  13
               ],
               "weight": 0.9
            },
            {
               "word": ".Redogörelsen",
               "text": "Redogörelsen uppdaterades senast den 17 juni 2025",
               "type": "redogörelse",
               "date": [
                  2025,
                  6,
                  17
               ],
               "weight": 0.9
            },
            {
               "word": ".Redogörelsen",
               "text": "Redogörelsen uppdaterades: 13 januari 2025",
               "type": "redogörelse",
               "date": [
                  2025,
                  1,
                  13
               ],
               "weight": 0.9
            },
            {
               "word": ".Redogörelsen",
               "text": "Redogörelsen uppdaterades senast den 13 januari 2025",
               "type": "redogörelse",
               "date": [
                  2025,
                  1,
                  1
               ],
               "weight": 0.9
            },
            {
               "word": ".Redogörelsen",
               "text": "Redogörelsen uppdaterades senast den 17 juni 2025",
               "type": "redogörelse",
               "date": [
                  2025,
                  6,
                  1
               ],
               "weight": 0.9
            },
            {
               "word": ".Redogörelsen",
               "text": "Redogörelsen uppdaterades: 13 januari 2025",
               "type": "redogörelse",
               "date": [
                  2025,
                  1,
                  1
               ],
               "weight": 0.9
            },
            {
               "word": "Uppdaterad:",
               "text": "Uppdaterad: 18 juni 2025",
               "type": "uppdater",
               "date": [
                  2025,
                  6,
                  18
               ],
               "weight": 0.5
            }
         ]
      }
   ],
   "visited-urls": {},
   "has-a11y-statement": true
}

7h3Rabbit avatar Jul 05 '25 22:07 7h3Rabbit

Tried v2025.7.3 right now with python default.py -t 26 -u https://www.vgregion.se -L sv -r -o vgregion-26.json and get score 5.0 even though the test fails.

Output file:
{ "tests": [ { "data": { "dependencies": { "@sitespeed.io/log": "0.2.6", "@sitespeed.io/plugin": "1.0.0" }, "groups": { "www.vgregion.se": { "dependencies": { "@sitespeed.io/log": "0.2.6", "@sitespeed.io/plugin": "1.0.0" }, "issues": [], "pages": { "https://www.vgregion.se": { "dependencies": { "@sitespeed.io/log": "0.2.6", "@sitespeed.io/plugin": "1.0.0" }, "issues": [], "score": { "overall": 100 }, "url": "https://www.vgregion.se", "version": "2025.6.0" } }, "score": { "overall": 100 }, "version": "2025.6.0" } }, "version": "2025.6.0" }, "date": "2025-07-23T00:24:29.105554", "rating": 5.0, "rating_a11y": -1.0, "rating_perf": -1.0, "rating_sec": -1.0, "rating_stand": -1.0, "report": "", "report_a11y": "", "report_perf": "", "report_sec": "", "report_stand": "", "site_id": 0, "type_of_test": -1 } ] }

Terminal error:
[2025-07-25 23:30:13] ERROR: TypeError: anchor.replace is not a function at file:///home/webperf/next_webperf_core_2025.7.3/node_modules/plugin-accessibility-statement/lib/harAnalyzer.js:484:83 at Proxy.forEach (<anonymous>) at HarAnalyzer.tryGetNotificationUrl (file:///home/webperf/next_webperf_core_2025.7.3/node_modules/plugin-accessibility-statement/lib/harAnalyzer.js:459:36) at HarAnalyzer.createKnowledgeFromData (file:///home/webperf/next_webperf_core_2025.7.3/node_modules/plugin-accessibility-statement/lib/harAnalyzer.js:149:55) at HarAnalyzer.analyzeData (file:///home/webperf/next_webperf_core_2025.7.3/node_modules/plugin-accessibility-statement/lib/harAnalyzer.js:732:42) at AccessibilityStatementPlugin.processMessage (file:///home/webperf/next_webperf_core_2025.7.3/node_modules/plugin-accessibility-statement/lib/index.js:92:43) at file:///home/webperf/next_webperf_core_2025.7.3/node_modules/sitespeed.io/lib/core/queueHandler.js:220:32 at tryProcessNext (file:///home/webperf/next_webperf_core_2025.7.3/node_modules/sitespeed.io/lib/core/queue.js:41:39) at Object.enqueue [as queue] (file:///home/webperf/next_webperf_core_2025.7.3/node_modules/sitespeed.io/lib/core/queue.js:70:5) at QueueHandler.postMessage (file:///home/webperf/next_webperf_core_2025.7.3/node_modules/sitespeed.io/lib/core/queueHandler.js:211:12) at BrowsertimePlugin.sendMessage (file:///home/webperf/next_webperf_core_2025.7.3/node_modules/sitespeed.io/node_modules/@sitespeed.io/plugin/plugin.js:110:23) at BrowsertimePlugin.processMessage (file:///home/webperf/next_webperf_core_2025.7.3/node_modules/sitespeed.io/lib/plugins/browsertime/index.js:604:21)

failures.log is empty for some reason.

marcusosterberg avatar Jul 25 '25 21:07 marcusosterberg