Reduce boilerplate for browser tests

[ka3de]

Currently, as of v0.46.0, browser tests require the browser.type option to be set. This can only be defined in a scenario definition such as:

export const options = {
  scenarios: {
    ui: {
      executor: 'shared-iterations',
      options: {
        browser: {
            type: 'chromium',
        },
      },
    },
  },
}

Which implies that this is the minimum required [boilerplate] code for a test to be able to use the browser module.

This is OK for a long test, or for tests that already make use of scenarios. But for shorter tests, and especially for "beginner" tests this implies an entry barrier for getting started with the product. E.g.: Requires the users to hassle with scenarios definition, including executor definition, which are more advanced features.

Therefore from the @grafana/k6-browser team we would like to understand and sync on which are the possible options to improve this and reduce the required boilerplate for browser tests.

Without any knowledge in the code implications a possible improvement would be to:

• For tests with no scenarios definition: Support a top level option definition to specify the browser type (possibly also support this through a cmdline arg, similar to -u for VUs):

export const options = {
  browser_type: 'chromium' // This must also be "parseable" from the k6 browser side
}

• For tests with scenarios definition: Keep the current schema for browser options definition inside scenarios.

[dgzlopes]

This is OK for a long test, or for tests that already make use of scenarios. But for shorter tests, and especially for "beginner" tests this implies an entry barrier for getting started with the product. E.g.: Requires the users to hassle with scenarios definition, including executor definition, which are more advanced features.

Yup, this. Also, for browser-only tests, it is a hassle to write all the scenarios boilerplate.

For tests with no scenarios definition: Support a top level option definition to specify the browser type (possibly also support this through a cmdline arg, similar to -u for VUs): export const options = { browser_type: 'chromium' // This must also be "parseable" from the k6 browser side }

In case a user configures both, what would happen? Right now, if you configure scenarios and then VUs, it will fail to run; we should probably mimic that behavior.

About the proposal: Would it make sense to keep the browser section?

export const options = {
    browser: {
        type: 'chromium',
    },
}

The main benefit I see is that I can use the same block with/without scenarios and don't have to learn anything new.

Also, even if atm we only support browser type, maybe in the future, we move more things in that block (e.g. headless).

[ankur22]

Also, even if atm we only support browser type, maybe in the future, we move more things in that block (e.g. headless).

I think we need to be careful which fields we add to the browser options object. If we allowed the setting of headless in the browser options then what should the behaviour be for a remote browser? Should it ignore the field (which could be confusing behaviour) or should it error and exit the test (not a good DX)?

In this issue are we also proposing a solution without the scenarios having to be defined? I.e. so that we could work with equivalent command line flags to -u or -v? Or should that be discussed in a separate issue?

[dgzlopes]

If we allowed the setting of headless in the browser options then what should the behaviour be for a remote browser? Should it ignore the field (which could be confusing behaviour) or should it error and exit the test (not a good DX)?

Totally fair. But the env var is also a terrible DX -> Maybe this one should be a flag?

In this issue are we also proposing a solution without the scenarios having to be defined? I.e. so that we could work with equivalent command line flags to -u or -v? Or should that be discussed in a separate issue?

IMO, it is all part of the same convo.

[inancgumus]

I liked @dgzlopes's suggestion: it aligns well with the scenario option definition, and users won't have to learn a new syntax.

This part always stays the same, even with or without scenarios:

browser: {
    type: 'chromium',
}

maybe in the future, we move more things in that block (e.g., headless).

Even if it's not for headless, we can put slowMo here. So this sounds like a good call 👍

[ppcano]

In addition to the previous suggestion, it could also support:

export const options = {
    browser: true 
}

This configuration should use all the default settings for browser testing.