k6
k6 copied to clipboard
Specifying --iterations or --duration on the command line makes it impossible to use `k6/experimental/browser`
Brief summary
Because of the behavior described in #3741, it's not possible to use k6/experimental/browser if you specify --iterations or --duration on the command line.
k6 version
v0.51.0
OS
Linux
Docker version and image (if applicable)
N/A
Steps to reproduce the problem
Use this script:
import { browser } from 'k6/experimental/browser';
export const options = {
scenarios: {
default: {
executor: 'shared-iterations',
options: {
browser: {
type: 'chromium',
},
},
},
},
thresholds: {
checks: ['rate==1.0'],
},
};
export default async function() {
const page = browser.newPage();
try {
await page.goto('https://test.k6.io/');
} finally {
page.close();
}
}
Run it:
$ k6 run ./test-3.js
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: ./test-3.js
output: -
scenarios: (100.00%) 1 scenario, 1 max VUs, 10m30s max duration (incl. graceful stop):
* default: 1 iterations shared among 1 VUs (maxDuration: 10m0s, gracefulStop: 30s)
browser_data_received.......: 27 kB 27 kB/s
browser_data_sent...........: 1.0 kB 1.0 kB/s
browser_http_req_duration...: avg=234.71ms min=100.5ms med=275.16ms max=328.49ms p(90)=317.82ms p(95)=323.15ms
browser_http_req_failed.....: 0.00% ✓ 0 ✗ 3
browser_web_vital_cls.......: avg=0.000074 min=0.000074 med=0.000074 max=0.000074 p(90)=0.000074 p(95)=0.000074
browser_web_vital_fcp.......: avg=625.9ms min=625.9ms med=625.9ms max=625.9ms p(90)=625.9ms p(95)=625.9ms
browser_web_vital_lcp.......: avg=625.9ms min=625.9ms med=625.9ms max=625.9ms p(90)=625.9ms p(95)=625.9ms
browser_web_vital_ttfb......: avg=325.7ms min=325.7ms med=325.7ms max=325.7ms p(90)=325.7ms p(95)=325.7ms
✓ checks......................: 0.00% ✓ 0 ✗ 0
data_received...............: 0 B 0 B/s
data_sent...................: 0 B 0 B/s
iteration_duration..........: avg=852.18ms min=852.18ms med=852.18ms max=852.18ms p(90)=852.18ms p(95)=852.18ms
iterations..................: 1 0.993069/s
vus.........................: 1 min=1 max=1
vus_max.....................: 1 min=1 max=1
running (00m01.0s), 0/1 VUs, 1 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs 00m01.0s/10m0s 1/1 shared iters
pass --iterations on the command line:
$ k6 run --iterations 1 ./test-3.js
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: ./test-3.js
output: -
scenarios: (100.00%) 1 scenario, 1 max VUs, 10m30s max duration (incl. graceful stop):
* default: 1 iterations shared among 1 VUs (maxDuration: 10m0s, gracefulStop: 30s)
ERRO[0000] Uncaught (in promise) GoError: browser not found in registry. make sure to set browser type option in scenario definition in order to use the browser module
at github.com/grafana/xk6-browser/browser.mapBrowser.func7 (native)
at file:///home/marcelo/devel/grafana/tmp/k6/test-3.js:20:15(4) executor=shared-iterations scenario=default
✓ checks...............: 0.00% ✓ 0 ✗ 0
data_received........: 0 B 0 B/s
data_sent............: 0 B 0 B/s
iteration_duration...: avg=86.62µs min=86.62µs med=86.62µs max=86.62µs p(90)=86.62µs p(95)=86.62µs
iterations...........: 1 5390.574042/s
running (00m00.0s), 0/1 VUs, 1 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs 00m00.0s/10m0s 1/1 shared iters
Expected behaviour
The script runs using the number of iterations specified on the command line.
Or it produces an error that makes sense (the current one says "make sure to set browser type option in scenario definition", which is there, from the point of view of the user).
Actual behaviour
The script doesn't run and the error that is emitted is confusing.
I'm guessing in the future it will make sense to specify different browser types in different scenarios, so it makes sense to have the setting at the scenario level.
That said, I'm also guessing it makes sense to have an environment variable specifying a global default value. (Environment variable because I don't think extensions can create additional command line options, can they?)
Another avenue would be to specify a global default as part of options outside of the scenarios (and tie this to the environment variable), and still use the per-scenario setting if that is needed.
(adding the ux label because to me this is UX; please remove if that's inappropriate for the current usage of that label)