browsertime icon indicating copy to clipboard operation
browsertime copied to clipboard

The parameter `--script` works before the test. But the documentation is about `collect metrics and run after the page has finished loading`

Open polarnik opened this issue 7 months ago • 9 comments

Have you read the documentation?

URL

https://youtrack.jetbrains.com

What are you trying to accomplish

Summary

sitespeedio/sitespeed.io:37.5.1-plus1

Steps

I read the docs about the param:

--browsertime.script, --script Add custom Javascript that collect metrics and run after the page has finished loading. Note that --script can be passed multiple times if you want to collect multiple metrics. The metrics will automatically be pushed to the summary/detailed summary and each individual page + sent to Graphite

and created a file /metrics.js with the content:

return (function() {
    console.log("metrics.js");
    try {
        const entries = performance.getEntriesByType("navigation");
        const entry = entries[0];

        console.log(`metrics.js: entry: ${entry}`);
        console.log(`metrics.js: entries: ${entries}`);
    } catch (e) {}
})();

and passes a view params to the test:

--browsertime.script /metrics.js

Actual result

I see metrics.js messages before the test, but not after the page has finished loading

Expected result

I will see metrics.js messages after the page has finished loading, or I will see an actual documentations about how to add custom measurements

What browser did you use?

Chrome

How to reproduce

--browsertime.script /metrics.js

Log output

STDOUT: Chromium 136.0.7103.25 
STDOUT: Mozilla Firefox 136.0.4
STDOUT: Architecture: aarch64
STDOUT: [2025-05-30 15:10:59] INFO: Versions OS: linux 6.10.14-linuxkit nodejs: v22.13.0 sitespeed.io: 37.5.1 browsertime: 24.6.0 coach: 8.1.1
STDOUT: [2025-05-30 15:10:59] INFO: The following plugins are enabled: assets, browsertime, budget, coach, domains, harstorer, html, lateststorer, metrics, pagexray, remove, text, thirdparty
STDOUT: metrics.js
STDOUT: metrics.js: entry: undefined
STDOUT: metrics.js: entries: 
STDOUT: [2025-05-30 15:10:59] INFO: Running tests using Chrome - 1 iteration(s)
STDOUT: [2025-05-30 15:11:00] INFO: Testing url https://youtrack.jetbrains.com/ iteration 1
STDOUT: [2025-05-30 15:11:11] INFO: Take after page complete check screenshot

polarnik avatar May 30 '25 15:05 polarnik

Hi @polarnik I think the documentation is very unclear then, let me check that this weekend. In your JavaScript you need to return the metric back to NodeJS process. If I remember correctly, the metric name will be the filename (without the .js) and then you need to return the metric, so for example "return 1" will give a 1.

soulgalore avatar May 30 '25 16:05 soulgalore

Here are some example that is used inside of Browsertime: https://github.com/sitespeedio/browsertime/tree/main/browserscripts/timings

soulgalore avatar May 30 '25 16:05 soulgalore

I have found a workaround

The param --browsertime.pageCompleteCheck /defaultPageCompleteCheck.js

The file /defaultPageCompleteCheck.js is a file https://github.com/sitespeedio/browsertime/blob/main/lib/core/pageCompleteChecks/defaultPageCompleteCheck.js but with some changes:

return (function(waitTime) {

    try {
        var end = window.performance.timing.loadEventEnd;
        var start= window.performance.timing.navigationStart;

        let status = (end > 0) && (performance.now() > end - start + waitTime);
        if(status) {
            // Custom metrics
            try {
                const entries = performance.getEntriesByType("navigation");
                const entry = entries[0];

                performance.measure( "0_startTime_loadEventEnd", {
                    start: entry.startTime,
                    duration: entry.loadEventEnd - entry.startTime,
                });

                performance.mark( "0_start_end", {
                    startTime: window.performance.timing.loadEventEnd - window.performance.timing.navigationStart,
                });
            } catch (e) {
                console.log(`defaultPageCompleteCheck: exception 1: \n${JSON.stringify(e)}`)
            }
            return true;
        } else {
            return false;
        }
    } 
    catch(e) {
        console.log(`defaultPageCompleteCheck: exception 2 : ${JSON.stringify(e)}`);
        return true;
    }
})(arguments[arguments.length - 1]);

polarnik avatar May 30 '25 16:05 polarnik

Ah sorry I see know what you mean, if it happens before it looks like a bug let me test it.

soulgalore avatar May 30 '25 16:05 soulgalore

I still don't fully understand, when you use --browsertime.script it will run JavaScript inside the browser. So when you do console.log it will log to the browser, not as in your example. Are you sure you don't run with --preScript or something like this?

soulgalore avatar May 30 '25 19:05 soulgalore

Are you sure you don't run with --preScript or something like this?

Yes, I'm sure. I used --browsertime.script /metrics.js and --browsertime.pageCompleteCheck /defaultPageCompleteCheck.js only

polarnik avatar May 31 '25 12:05 polarnik

Ok. I will try to check:

Different paths:

  • --browsertime.script /metrics.js
  • --browsertime.script /custom/metrics.js
  • --browsertime.script /timings/metrics.js

Paths without .js:

  • --browsertime.script /metrics
  • --browsertime.script /custom/metrics
  • --browsertime.script /timings/metrics

Folders:

  • --browsertime.script /
  • --browsertime.script /custom/
  • --browsertime.script /timings/
  • --browsertime.script /custom
  • --browsertime.script /timings

Scripts with different contents

  • export function ...
  • return 1
  • 1
  • return 1;
  • 1;
  • return (function() { })();
  • return (function() { console.log("test_xxx") })();
  • return (function() { performance.measure(text_xxx,{ start: 1337, duration: 7331, }); })();
  • (function() { })();
  • return (function() { return 1; })();

About my prev tries:

  • scripts with return inside didn't work because of some JS syntax error
    • return (function() { performance.measure(text_xxx,{ start: 1337, duration: 7331, }); })();
  • but scripts without return as the first word didn't create the new metric
    • (function() { performance.measure(text_xxx,{ start: 1337, duration: 7331, }); })();

I will try other options

polarnik avatar May 31 '25 12:05 polarnik

By the way, this workaround with --browsertime.pageCompleteCheck /defaultPageCompleteCheck.js works really well for me https://github.com/sitespeedio/browsertime/issues/2296#issuecomment-2922851818

Image

I have a lot of custom metrics

polarnik avatar May 31 '25 12:05 polarnik

The metrics you show there are user timings, they are automatically picked up.

Can you show me exactly the full command line you use?

If I create a metric like this in a file called mySuperMetric.js:

(function () {
  return 1;
})();

And then run it like this: sitespeed.io --browsertime.script mySuperMetric.js https://www.sitespeed.io -n 1 -o

My metric ends up here:

Image

soulgalore avatar Jun 01 '25 04:06 soulgalore