Did lighthouse CI changed criteria after Jun 23?

[aloxe]

I installed Lighthouse CI on my project workflow with a simple configuration It was using @lhci/[email protected] and was triggered on every PR.

From 23 June, it started to report errors, not passing the PR. Reverting changes and versions did not solve the issue. In fact manually running he lighthouse CI on the exact same code base is now reporting errors. Some errors do not make sense with NaN scores.

   ✘  lcp-lazy-loaded failure for minScore assertion
       Largest Contentful Paint image was not lazily loaded
       https://web.dev/articles/lcp-lazy-loading
       Audit did not produce a value at all. "minScore" might not be a valid assertion for this audit.

        expected: >=0.9
           found: NaN
      all values: NaN

Last lighthouse CI with OK results

Same lighthouse CI on the same codebase with KO results

The codebase and the github utilities remained untouched between the two. So I am wondering if thiere was not a change on the lighthouse side that could have triggered this change.

[boromir674]

Very important. have similar issue with my CI starting to report Nan values, such as:

  {
    "name": "minScore",
    "expected": 0.9,
    "actual": null,
    "values": [
      null,
      null,
      null
    ],
    "operator": ">=",
    "passed": false,
    "message": "Audit did not produce a value at all. \"minScore\" might not be a valid assertion for this audit.",
    "auditId": "lcp-lazy-loaded",
    "level": "error",
    "url": "http://localhost:46869/index.html",
    "auditTitle": "Largest Contentful Paint image was not lazily loaded",
    "auditDocumentationLink": "https://web.dev/articles/lcp-lazy-loading"
  }

CI: https://github.com/boromir674/konstantinoslampridis.io/actions/runs/17386412812/job/49353589964

I am using lhci 0.14.x:

  - run: yarn global add @lhci/[email protected]

In my case what is even more "scary" is that lhci exhibits this behavior (with null values being reported in Audits) only on the CI, but I cannot reproduce them locally.