CLS field value difference Web Vitals JS and Google page speed test

[shailenderjain]

Hello,

I am using the web-vitals.attribution.js to capture the field value for CLS, FID and LCP. However, there is a distinct difference between the values captured by the JS library vs "Google page speed test" field values. I am using the following code to capture the field value.

<script>
function sendToGoogleAnalytics ({name, delta, value, id, attribution}) {
  const eventParams = {
    // Built-in params:
  event_category: 'Web Vitals',
    value: delta, // Use `delta` so the value can be summed.
    // Custom params:
    metric_id: id, // Needed to aggregate events.
    metric_value: value, // Optional.
    metric_delta: delta, // Optional.
  }

  switch (name) {
    case 'CLS':
      eventParams.event_label = attribution.largestShiftTarget;
      //console.log("cls: ", attribution.largestShiftTarget);
      //console.log("cls source: ",	attribution.largestShiftSource);
      break;

    case 'FID':
      eventParams.event_label = attribution.eventTarget;
      // console.log("FID: ", attribution.eventTarget);
      break;

    case 'LCP':
      eventParams.event_label = attribution.element;
      //console.log("LCP: ", attribution.element);
      break;
  }

  // Assumes the global `gtag()` function exists, see:
  // https://developers.google.com/analytics/devguides/collection/ga4
  gtag('event', name, eventParams);
}
</script>


<!-- Append the `?module` param to load the module version of `web-vitals` -->
<script type="module">
  import {onCLS, onFID, onLCP} from 'https://unpkg.com/web-vitals@3/dist/web-vitals.attribution.js?module';
  onCLS(sendToGoogleAnalytics);
  onFID(sendToGoogleAnalytics);
  onLCP(sendToGoogleAnalytics);

  //onCLS(console.log);
  //onFID(console.log);
  //onLCP(console.log);
</script>

[philipwalton]

Sorry, it's not really clear what your question is, can you provide more details?

As to why the data captured by web-vitals might be different from the data exposed in PSI, see this document: https://web.dev/crux-and-rum-differences/

[shailenderjain]

I have attached 2 images for comparison. I have used Web Vitals for real time user monitoring. The CLS value is 0. However, when i run the same page on "Google page speed test" i can see the field value indicate 80% with 0. This is a gap between chrome measurement and Web Vitals library.

[philipwalton]

Do you have any iframes on your pages?

[shailenderjain]

No ...i do not have any iframes on my pages. But i have added Google adsense on the pages.

[shailenderjain]

There is one more observation. I was trying to drill down the CLS. I tried to capture attribution.largestShiftTarget. I could see the maximum value is coming as "(not set)". What is the significance of "(not set)".

[philipwalton]

Thanks, that screenshot is useful and I think I may know what's going on.

What is the significance of "(not set)".

"(not set)" is a value Google Analytics displays when you haven't supplied your own a value for a dimension. (It's also possible you manually set it to "(not set)", though I don't see that anywhere in the code you provided.)

If you're seeing "(not set)" that should mean that for those events, there was no largestShiftTarget value, which would only happen if there were no layout shifts. That means that for all the other events where "Event Label" is set (e.g. html>body>div.bannerdiv), there were layout shifts, despite the fact that the value is showing as 0.

So since I can infer from this report that there were layout shifts but the event value is showing as zero, that makes me suspect that you're not multiplying your CLS value by 1000 to convert it to an integer (as shown in these instructions).

In Universal Analytics (which I can tell you're using from the screenshot), even values must be integers, so if the CLS value that you're reporting for your page is 0.123, then my guess is it's just either dropping it altogether or rounding it to 0.

If you had upgraded to GA4, then you could sending float values for events, but you'd have to be using GA4 to do that.

[shailenderjain]

Thanks.. for your response. This is very helpful, However, i still need to understand the following.

1. As per the Web Vitals report there were 7697 unique events and 7411 did not experience any CLS. This mean the users with no CLS will be approx. 96%
2. However, from google page speed test the number of users with 0 or no CLS is approx. 80%.
3. This means as per the web vitals library the % of users with CLS will be approx. 4%. Whereas as google page speed it should be approx. 20%
4. I have been trying to understand this gap. I understand chrome uses some sampling method. But the difference is huge.
5. Additionally, as per my observation the gap between Web Vitals library and Google page speed test started occurring from June 21.

[tunetheweb]

I presume you are filtering your Google Analytics to Desktop or Mobile to match PageSpeed Insights? And more specifically Chrome Desktop or Mobile users (so ignoring other Chromium users that also expose CLS such as Edge, or in-app WebViews)? And that you are also looking over the 28-days that CrUX collects over?

[shailenderjain]

Thanks...I did the calculation according to the filter mentioned by you.

1. 5669 unique events with no cls
2. 139 unique events with CLS (Filter = Chrome, Desktop).
3. This means approx. 2.4% of the users should experience CLS. However, the same is reported as 20%

[philipwalton]

Quick clarification: the PageSpeed Insights report you posted shows that 80% of visits have good CLS, not that they have no CLS. Good CLS is defined as having a score of <= 0.1.

My guess is that many of cases reported via your attribution data will show that they have some layout shifting happening, but it's very small and well below the 0.1 "good" threshold. However, you'll have to update your code to fix the reporting issue in order to verify that.

[shailenderjain]

I have got another page where i have multiplied CLS with 1000.

Filter = Chrome & Desktop Total Events = 68186 Events (not set) = 66855 The events which have some attribution data shows an average value less then <0.01 Hence, the approx. % of users with 0 CLS should be around 98%. However, page speed test show 82%.

I have been trying to do a research on the CLS using Web Vitals from a long time. My thinking is again something changed around Jun 21 which had created a discrepancy between the Web Vitals data and Chrome reported data.

My another observation is that this could be due to Google Ad sense. I have enabled this on my site from last 3 years. But, again something changed in last year Jun 21. I have only enabled Vignette ads. But, surprisingly this is creating CLS.

Another point to note is that the issue with CLS only occurs on Desktop with Vignette Ads. This does not occur on Mobile. My CLS on mobile is 100%.

But the issue is why Web Vitals does not capture CLS?

[tunetheweb]

When did you set the multiplication by 1000?

PSI will show the values over the last 28-days so if you only set it recently in GA, it's not surprising they do not match.

It stills seems very unlikely that none of your page views have registered a CLS greater than 0. Which to me suggests this still isn't being tracked correctly. Even a perfect page can generate CLS - for example I use Chrome's translate option on that page and get a small CLS of 0.01037 when translating to English. Some changes like this, or due to extensions, may be out of the site owners control but usually represent a very small minority but none-the-less it's very unlikely this sort of thing is not represented in your analytics at all.

In fact when I do that using the Google Analytics extension on that page I can see the CLS is not being multiplied by 1000 and is still therefore registering as zero:

And looking at the code on that page I also don't see it multipled by 1000, except for one commented out console.log line:

function sendToGoogleAnalytics({name, delta, value, id, attribution}) {
--
  | const eventParams = {
  | // Built-in params:
  | event_category: 'Web Vitals PDF ID',
  | value: delta, // Use `delta` so the value can be summed.
  | // Custom params:
  | metric_id: id, // Needed to aggregate events.
  | metric_value: value, // Optional.
  | metric_delta: delta, // Optional.
  | }
  |  
  | switch (name) {
  | case 'CLS':
  | eventParams.event_label = attribution.largestShiftTarget;
  | //console.log("cls: ", attribution.largestShiftTarget, ":", parseFloat(delta)*1000);
  | //console.log("cls source: ",	attribution.largestShiftSource);
  | break;

[shailenderjain]

Thanks... a lot... I have updated my code. May be there was issue with my CDN. I will try to update outcome after 5 days.

[shailenderjain]

This is the result of last 5 days. This is after updating code with multiplication of 1000.

1. Approx. 9298 unique events
2. 175 unique events with attribution values

It looks like only 2% had some layout shift. This is inline with previous result. Wherever there is no attribution node the value for CLS is very low. This means at least approx. 98% should have CLS less then 0.1. However, google page speed test is still at 82%.

[tunetheweb]

You are still not sending integer values so think they still won't be read by GA properly and imagine all but exact integers are being ignored and treated as zero.

We note in the README that you should use Math.round (but AFTER multipling by 1000).

    // Google Analytics metrics must be integers, so the value is rounded.
    // For CLS the value is first multiplied by 1000 for greater precision
    // (note: increase the multiplier for greater precision if needed).
    eventValue: Math.round(name === 'CLS' ? delta * 1000 : delta),

[Anando12]

OPPO F15

On Wed, Oct 12, 2022, 14:56 Barry Pollard @.***> wrote:

You are still not sending decimal values so think they still won't be read by GA properly and imagine all but exact integers are being ignored and treated as zero.

We note in the README https://github.com/GoogleChrome/web-vitals#using-analyticsjs that you should use Math.round (but AFTER multipling by 1000).

// Google Analytics metrics must be integers, so the value is rounded.
// For CLS the value is first multiplied by 1000 for greater precision
// (note: increase the multiplier for greater precision if needed).
eventValue: Math.round(name === 'CLS' ? delta * 1000 : delta),

— Reply to this email directly, view it on GitHub https://github.com/GoogleChrome/web-vitals/issues/263#issuecomment-1275824271, or unsubscribe https://github.com/notifications/unsubscribe-auth/APRHNQP6PRGM26QBWJPMSQTWCZ4LZANCNFSM6AAAAAAQXUDAJA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[shailenderjain]

ok.. I have changed the code to Math.round(delta*1000)

[shailenderjain]

Here are the result of the last 6 days. 10533 out of 10749 have an almost 0 CLS. This means at least 97.9 have less then 0.1

Trying to drill down on the second row item which is showing value of 704, I could see there were few users with some large values.

Overall, if the right CLS value is derived after division with 1000, then i cannot see any user with CLS more then 0.1. Almost 100% users have CLS less then 0.1. Whereas google page speed test is at 84%

[tunetheweb]

I don't have an answer I'm afraid, but I do have a few comments:

The unique events don't really matter for this report. If two people experience the same shift (or one person experiences the same shift twice on loading the page), then that still counts as two people, even if it's one unique event. Unique views make more sense for page views. But still your unique events and total events are nearly similar (in fact total looks even better in terms of CLS) but thought worth noting.

You cache your page for nearly 4 days (333200 seconds). I wonder if you are still getting a significant number of page views for the code that wasn't sending back the right values (x 1000)? I wonder if it will change over time and get closer to CrUX?

You are sending back two CLS events - one is correctly multiplied up by 1000, one is not. I presume in above screenshots you are only looking at the correct one and not both?

You are sending a CLS event on each page visibility change. There should only be one event so you may be incorrectly counting multiple values. Is this why you are trying to use unique events?

You are not including events when the bfcache is used, but those will be measured in CrUX. This section details how to send events on those restores as well. But those should be relatively low CLS so doubt that's the cause.

It is interesting that you only experience the CLS issue in CrUX on desktop and not mobile. That suggests it may not be something wrong with the page, but more likely extensions, or other browser features (I notice when I use Chrome's built in translation tool causes a CLS, similarly when I zoom in). If so, these may be out of your control to a large part.

I also note that when I zoom in, the even fails to get sent with the error Payload size is too large (17241). Max allowed is 8192. as it tries to include basically the whole page as the event label. That may be skewing your numbers a bit if some of the events can't be logged.

The good news is that at 84%, you are well above the threshold, and the point of using the p75 value is in somewhat due to the fact that it is not always possible to get a perfect score. A CLS of 0 is always better of course, but there's a lots of varied users out there! However, it still seems odd to me that there is such a high percentage (16%) difference between your analytics and CrUX, so take above comments on board to see if you can narrow that down. Plus the page doesn't really show any shifts to me other than the translation issue.

[shailenderjain]

Thanks... a lot for taking time out and providing response. This is very helpful

From analytics i could see 10,916 session during the same period for the same page. This is almost equal to unique events. I had done the cache clearance from CDN. I think cache may not be issue.

2 CLS events are sent with different event_category. Hence, there should not be any overlap.

There is something related to translation. I understand many users may land on the page with browser configured as en-us and page language as id. I am not sure if this will always lead to translation which can cause CLS. Is the CLS related to translation not captured.

Another important point which i have observed. I have very similar page https://miniimagesvideos.com/id_compress_jpeg This page has CLS of 95% on desktop. The important point to note is this page has same layout, users from same country, almost same type of desktop.

[tunetheweb]

Is the CLS related to translation not captured.

No, I can see this IS captured. I can see GA events firing when I translate the page. I was just pointing out that things like this are often the cause of CLS rather than necessarily anything wrong with the page.