Async APIs

[sniku]

What

This issue has been repurposed to initiate and follow the progress of migrating most of the browser APIs to async (promise based).

A recent example is this issue where touchscreen.tap was migrated to async:

From:

page.touchscreen.tap(...);

To:

await page.touchscreen.tap(...);

Why

1. Most browser APIs use some form of long-running IO operation (networking) to perform the requested action on the web browser against the website under test. We need to avoid blocking JavaScript's runtime event loop for such operations.
2. We're going to add more asynchronous event-based APIs (such as page.on) that our current synchronous APIs would block.
3. To align with how developers expect to work with JavaScript APIs.
4. To have better compatibility with Playwright.

How

We have detailed exactly which APIs will most likely need to be migrated across in the comment within this issue. For most of the APIs we can take a look at the comment and migrate the APIs that are listed there. It's worth noting that it's still best to double check whether the API needs to be migrated especially ones which k6 browser implements but aren't provided by Playwright, for example page.throttleCPU.

NOT Implemented In PW

If an API is not implemented in PW, then you should be able to answer yes to one of the questions in the list below to determine whether it should be async or not, for example browser.context is not in PW, and it doesn't do any IO or long running tasks, therefore it can stay as a synchronous API.

1. Does the API perform a network call or other IO operations, this includes a CDP request to Chrome?
2. Does the API perform long running task?

Implemented in PW

Is the API in the list in this comment and is implemented in PW?

If the answer is yes, then it can be migrated to async.

Goja, docs and type defs

Initially we're only interested in moving the API to async, and not worrying about goja, doc and type definitions updates. Doc and type definition updates will be deferred to after the implementation work and before the release. We're hoping that the goja refactoring/abstraction work can also be shifted to a later point, but we may find that some APIs will also need goja to be refactored out of them to prevent panics and race conditions.

Original Description

We've decided to add Async API support to commonly used APIs. After adding the barebones, we can start adding the Async API support to the rest of the Page and ElementHandle methods.

Here's a list of tasks for that matter:

• [ ] BrowserType.Launch
• [ ] Browser.NewContext
• [ ] BrowserContext.NewPage
• [ ] Browser.Close
• [ ] Close
• [ ] Query
• [ ] QueryAll
• [ ] WaitForSelector
• [ ] WaitForFunction
• [ ] WaitForLoadState
• [ ] Type
• [ ] Evaluate — This will allow users to use Async support for the missing Async APIs until we fully support the API.
• [ ] Screenshot
• [x] #463
• [x] #439
• [x] #440
• [x] #583

Async Locator API tasks:

TBA

We can start with WaitForNavigation and Click (the ones we get the most problems) — I reordered them in the task list.

Note

Async support should also be added to the inner types. For example, Page.Type calls ElementHandle.Type and so on. Adding Async support to the Locator API can help too.

Pitfalls

Currently (v0.39), k6 does not support the await keyword. The problem stems from the fact that goja doesn't support the async and await keywords yet.

We need to use then and catch blocks. This might be difficult for some users as they can't easily translate Playwright into xk6-browser. Both Playwright and Puppeteer use async.

For example, we can't type the following:

const response = await page.click("a");

We need to do it as follows:

const response = Symbol();

page.click("a")
    .then(v => {
        response = v;
    })
    .catch(v => {
         // ...
    })

Let us know if you have a better idea. This could become more complicated when we use a chain of promises.

About page.waitForXXX

• Puppeteer needs waitForNavigation() and waitForSelector to explicitly wait after a click.
• Playwright automatically handles that: Navigating to a URL auto-waits for the page to fire the load event.
• Playwright: In lazy loaded pages, it's still useful to wait for an element, see.
• We have a similar auto-waiting mechanism. However, it's unclear whether async support will allow us to use response without the waitFor methods.

// Puppeteer
const [response] = await Promise.all([
  page.waitForNavigation(),
  page.click("a")
]);
// use response here or do action on an element in the navigated page

// Playwright
response = await page.click("a");
// use response here

Also, for Playwright, clicking an element could trigger asynchronous processing before initiating the navigation or trigger multiple navigations. In these cases, it is recommended to call page.waitForNavigation explicitly.

// Promise.all prevents a race condition between clicking and waiting for a navigation.
await Promise.all([
  // Waits for the next navigation.
  // It is important to call waitForNavigation before click to set up waiting.
  page.waitForNavigation(),
  // Triggers a navigation after a timeout.
  page.locator('div.delayed-navigation').click(),
]);

### Migrate to Async
- [ ] #1302
- [ ] #1303
- [ ] #1307
- [ ] #1308
- [ ] #1309
- [ ] xk6-browser: Migrate `main-async` into `main`
- [ ] k6-docs: Migrate `main-browser-async` into `main`
- [ ] k6-definitelyTyped: Migrate `master-browser-async` into the next release branch
- [ ] k6: Migrate `main-browser-async` into the next release branch

### Document the changes
- [ ] #1298
- [ ] #1321
- [ ] #1294
- [ ] #1295
- [ ] #1305
- [ ] #1304
- [ ] #1306
- [ ] #1296

### Done Done
- [ ] #683
- [ ] #463
- [ ] #439
- [ ] #440
- [ ] #583
- [ ] #1251

[inancgumus]

This is the first plan. Please let me know what you think. @imiric @ankur22 @sniku

[ankur22]

I believe this is an up to date list of APIs that we should move over to be asynchronous (i.e. wrap the return in a Promise). The list was created by comparing the return types between Playwright's and k6's type definitions files.:

Class	Method	K6 Return	PW Return
Browser	closeContext	void	not found in Playwright
Browser	context	BrowserContext	not found in Playwright
Browser	newContext	BrowserContext	Promise<BrowserContext>
Browser	newPage	Page	Promise<Page>
Browser	userAgent	string	not found in Playwright
BrowserContext	addCookies	void	Promise
BrowserContext	addInitScript	void	not found in Playwright
BrowserContext	browser	Browser	null|Browser
BrowserContext	clearCookies	void	Promise
BrowserContext	clearPermissions	void	Promise
BrowserContext	close	void	Promise
BrowserContext	cookies	Cookie[]	Promise<Array<Cookie>>
BrowserContext	grantPermissions	void	Promise
BrowserContext	newPage	Page	Promise<Page>
BrowserContext	setGeolocation	void	Promise
BrowserContext	setOffline	void	Promise
ElementHandle	$	ElementHandle|null	Promise<ElementHandle<SVGElement|HTMLElement>|null>
ElementHandle	$$	ElementHandle[]	Promise<ElementHandle<SVGElement|HTMLElement>[]>
ElementHandle	boundingBox	Rect	Promise<null|{x:number;y:number;width:number;height:number;}>
ElementHandle	check	void	Promise
ElementHandle	contentFrame	Frame	Promise<null|Frame>
ElementHandle	dblclick	void	Promise
ElementHandle	dispatchEvent	void	Promise
ElementHandle	fill	void	Promise
ElementHandle	focus	void	Promise
ElementHandle	getAttribute	string|null	Promise<null|string>
ElementHandle	hover	void	Promise
ElementHandle	innerHTML	string	Promise
ElementHandle	innerText	string	Promise
ElementHandle	inputValue	string	Promise
ElementHandle	isChecked	boolean	Promise
ElementHandle	isDisabled	boolean	Promise
ElementHandle	isEditable	boolean	Promise
ElementHandle	isEnabled	boolean	Promise
ElementHandle	isHidden	boolean	Promise
ElementHandle	isVisible	boolean	Promise
ElementHandle	ownerFrame	Frame	Promise<null|Frame>
ElementHandle	press	void	Promise
ElementHandle	screenshot	ArrayBuffer	Promise<Buffer>
ElementHandle	scrollIntoViewIfNeeded	void	Promise
ElementHandle	selectOption	string[]	Promise<Array>
ElementHandle	selectText	void	Promise
ElementHandle	setInputFiles	void	Promise
ElementHandle	tap	void	Promise
ElementHandle	textContent	string	Promise<null|string>
ElementHandle	type	void	Promise
ElementHandle	uncheck	void	Promise
ElementHandle	waitForElementState	void	Promise
ElementHandle	waitForSelector	ElementHandle	Promise<null|ElementHandle<SVGElement|HTMLElement>>
Frame	$	ElementHandle|null	Promise<ElementHandle<SVGElement|HTMLElement>|null>
Frame	$$	ElementHandle[]	Promise<ElementHandle<SVGElement|HTMLElement>[]>
Frame	check	void	Promise
Frame	content	string	Promise
Frame	dblclick	void	Promise
Frame	dispatchEvent	void	Promise
Frame	evaluate<R, Arg>	R	Promise<R>
Frame	evaluateHandle<R, Arg>	JSHandle<R>	Promise<SmartHandle<R>>
Frame	fill	void	Promise
Frame	focus	void	Promise
Frame	frameElement	ElementHandle	Promise<ElementHandle>
Frame	getAttribute	string	Promise<null|string>
Frame	hover	void	Promise
Frame	innerHTML	string	Promise
Frame	innerText	string	Promise
Frame	inputValue	string	Promise
Frame	isChecked	boolean	Promise
Frame	isDisabled	boolean	Promise
Frame	isEditable	boolean	Promise
Frame	isEnabled	boolean	Promise
Frame	isHidden	boolean	Promise
Frame	isVisible	boolean	Promise
Frame	press	void	Promise
Frame	selectOption	string[]	Promise<Array>
Frame	setContent	void	Promise
Frame	setInputFiles	void	Promise
Frame	tap	void	Promise
Frame	textContent	string	Promise<null|string>
Frame	title	string	Promise
Frame	type	void	Promise
Frame	uncheck	void	Promise
Frame	waitForLoadState	void	Promise
Frame	waitForSelector	ElementHandle	Promise<null|ElementHandle<SVGElement|HTMLElement>>
Frame	waitForTimeout	void	Promise
JSHandle	asElement	ElementHandle|null	TextendsNode?ElementHandle<T>:null
JSHandle	dispose	void	Promise
JSHandle	evaluate<R, Arg>	R	not found in Playwright
JSHandle	evaluateHandle<R, Arg>	JSHandle<R>	not found in Playwright
JSHandle	getProperties	Map<string,JSHandle>	Promise<Map<string,JSHandle>>
JSHandle	jsonValue	any	Promise<T>
Keyboard	down	void	Promise
Keyboard	insertText	void	Promise
Keyboard	press	void	Promise
Keyboard	type	void	Promise
Keyboard	up	void	Promise
Locator	check	void	Promise
Locator	clear	void	Promise
Locator	dblclick	void	Promise
Locator	dispatchEvent	void	Promise
Locator	fill	void	Promise
Locator	focus	void	Promise
Locator	getAttribute	string|null	Promise<null|string>
Locator	hover	void	Promise
Locator	innerHTML	string	Promise
Locator	innerText	string	Promise
Locator	inputValue	string	Promise
Locator	isChecked	boolean	Promise
Locator	isDisabled	boolean	Promise
Locator	isEditable	boolean	Promise
Locator	isEnabled	boolean	Promise
Locator	isHidden	boolean	Promise
Locator	isVisible	boolean	Promise
Locator	press	void	Promise
Locator	selectOption	string[]	Promise<Array>
Locator	tap	void	Promise
Locator	textContent	string	Promise<null|string>
Locator	type	void	Promise
Locator	uncheck	void	Promise
Locator	waitFor	void	Promise
Mouse	click	void	Promise
Mouse	dblclick	void	Promise
Mouse	down	void	Promise
Mouse	move	void	Promise
Mouse	up	void	Promise
Page	$	ElementHandle|null	Promise<ElementHandle<SVGElement|HTMLElement>|null>
Page	$$	ElementHandle[]	Promise<ElementHandle<SVGElement|HTMLElement>[]>
Page	bringToFront	void	Promise
Page	check	void	Promise
Page	close	void	Promise
Page	content	string	Promise
Page	dblclick	void	Promise
Page	dispatchEvent	void	Promise
Page	emulateMedia	void	Promise
Page	emulateVisionDeficiency	void	not found in Playwright
Page	evaluate<R, Arg>	R	Promise<R>
Page	evaluateHandle<R, Arg>	JSHandle<R>	Promise<SmartHandle<R>>
Page	fill	void	Promise
Page	focus	void	Promise
Page	getAttribute	null|string	Promise<null|string>
Page	hover	void	Promise
Page	innerHTML	string	Promise
Page	innerText	string	Promise
Page	inputValue	string	Promise
Page	isChecked	boolean	Promise
Page	isDisabled	boolean	Promise
Page	isEditable	boolean	Promise
Page	isEnabled	boolean	Promise
Page	isHidden	boolean	Promise
Page	isVisible	boolean	Promise
Page	on	void	this
Page	opener	Page|null	Promise<null|Page>
Page	press	void	Promise
Page	reload	null|Response	Promise<null|Response>
Page	screenshot	ArrayBuffer	Promise<Buffer>
Page	selectOption	string[]	Promise<Array>
Page	setContent	void	Promise
Page	setExtraHTTPHeaders	void	Promise
Page	setInputFiles	void	Promise
Page	setViewportSize	void	Promise
Page	tap	void	Promise
Page	textContent	string	Promise<null|string>
Page	throttleCPU	void	not found in Playwright
Page	throttleNetwork	void	not found in Playwright
Page	title	string	Promise
Page	type	void	Promise
Page	uncheck	void	Promise
Page	viewportSize	{width:number;height:number;}	null|{width:number;height:number;}
Page	waitForLoadState	void	Promise
Page	waitForSelector	ElementHandle	Promise<null|ElementHandle<SVGElement|HTMLElement>>
Page	waitForTimeout	void	Promise
Request	allHeaders	Record<string,string>	Promise<{[key:string]:string;}>
Request	headerValue	string|null	Promise<null|string>
Request	headersArray	Array<{name:string;value:string}>	Promise<Array<{name:string;value:string;}>>
Request	postData	string	null|string
Request	resourceType	ResourceType	string
Request	response	Response|null	Promise<null|Response>
Request	size	{body:number;headers:number}	not found in Playwright
Request	timing	ResourceTiming	{startTime:number;domainLookupStart:number;domainLookupEnd:number;connectStart:number;secureConnectionStart:number;connectEnd:number;requestStart:number;responseStart:number;responseEnd:number;}
Response	allHeaders	Record<string,string>	Promise<{[key:string]:string;}>
Response	body	ArrayBuffer	Promise<Buffer>
Response	headerValue	string|null	Promise<null|string>
Response	headerValues	string[]	Promise<Array>
Response	headersArray	Array<{name:string;value:string}>	Promise<Array<{name:string;value:string;}>>
Response	json	any	Promise<Serializable>
Response	securityDetails	SecurityDetailsObject|null	Promise<null|{issuer?:string;protocol?:string;subjectName?:string;validFrom?:number;validTo?:number;}>
Response	serverAddr	{ipAddress:string;port:number}|null	Promise<null|{ipAddress:string;port:number;}>
Response	size	{body:number;headers:number}	not found in Playwright
Touchscreen	tap	void	Promise