[BUG] some getByRole roles don't work

[eatonjl]

System info

• Playwright Version: 1.33
• Operating System: Mac OS
• Browser: Chromium
• Other info:

Source code

• [x] I provided exact source code that allows reproducing the issue locally.

Config file

import { defineConfig, devices } from '@playwright/test';

/**
 * Read environment variables from file.
 * https://github.com/motdotla/dotenv
 */
// require('dotenv').config();

/**
 * See https://playwright.dev/docs/test-configuration.
 */
export default defineConfig({
  testDir: './tests',
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: 'html',
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Base URL to use in actions like `await page.goto('/')`. */
    // baseURL: 'http://127.0.0.1:3000',

    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on',
  },

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },

    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },

    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },

    /* Test against mobile viewports. */
    // {
    //   name: 'Mobile Chrome',
    //   use: { ...devices['Pixel 5'] },
    // },
    // {
    //   name: 'Mobile Safari',
    //   use: { ...devices['iPhone 12'] },
    // },

    /* Test against branded browsers. */
    // {
    //   name: 'Microsoft Edge',
    //   use: { ...devices['Desktop Edge'], channel: 'msedge' },
    // },
    // {
    //   name: 'Google Chrome',
    //   use: { ..devices['Desktop Chrome'], channel: 'chrome' },
    // },
  ],

  /* Run your local dev server before starting the tests */
  // webServer: {
  //   command: 'npm run start',
  //   url: 'http://127.0.0.1:3000',
  //   reuseExistingServer: !process.env.CI,
  // },
});

Test file (self-contained)

import { test, expect } from '@playwright/test';

test('getByRoleIssue', async ({ page }) => {
  await page.goto('https://www.google.com/search?q=pizza+hut&oq=pizza+hut&aqs=chrome..69i57.1522j0j9&sourceid=chrome&ie=UTF-8');
  await page.getByRole('link', { name: 'Order Online', exact: true }).click();
  await page.getByRole('caption', { name: 'See More'}).click();
});

Steps

• Place a debug step on the line "await page.getByRole('caption', { name: 'See More'}).click();"
• When the test hits the debug line, go to the page, inspect it, and add "role=caption" to the "See More" link on the page
• Continue the test

Next Do the same thing but add "role=button" instead

Expected

both tests should pass

Actual

The first one (with "caption") fails while the second one (with "button") passes. For some reason, captions don't work even though they are listed as an option in the getbyrole description. I have gotten caption to work when I only look by role. But if I look by role AND name, it fails.

[eatonjl]

On line 2615 of the types.d.ts file it says, "getByRole(role: "alert"|"alertdialog"|"application"|"article"|"banner"|"blockquote"|"button"|"caption"|"cell"|"checkbox"|"code"|"columnheader"|"combobox"|"complementary"|"contentinfo"|"definition"|"deletion"|"dialog"|"directory"|"document"|"emphasis"|"feed"|"figure"|"form"|"generic"|"grid"|"gridcell"|"group"|"heading"|"img"|"insertion"|"link"|"list"|"listbox"|"listitem"|"log"|"main"|"marquee"|"math"|"meter"|"menu"|"menubar"|"menuitem"|"menuitemcheckbox"|"menuitemradio"|"navigation"|"none"|"note"|"option"|"paragraph"|"presentation"|"progressbar"|"radio"|"radiogroup"|"region"|"row"|"rowgroup"|"rowheader"|"scrollbar"|"search"|"searchbox"|"separator"|"slider"|"spinbutton"|"status"|"strong"|"subscript"|"superscript"|"switch"|"tab"|"table"|"tablist"|"tabpanel"|"term"|"textbox"|"time"|"timer"|"toolbar"|"tooltip"|"tree"|"treegrid"|"treeitem", options?: {" You can see that "caption" is on the list.

[thomas-phillips-nz]

From my testing, the name property doesn't match when the role is set to "caption". Setting the locator to await page.getByRole('caption').click(); seems to work fine, so Playwright can find the element okay.

That said, what are you trying to achieve? You're explicitly adding role="caption" to the DOM, are you having issues with other caption locators? How does Playwright recognise them when using the "Pick locator" option in vscode?

For what it's worth: https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Roles

[eatonjl]

Thanks for looking into it! The reason I am explicitly adding role=caption is purely for re-creation purposes. The code I am working on is not easily sharable and I didn't want to spend hours trying to find a site that had an element with role=caption. I figured this was the easiest way to reproduce the issue. I have code in our application that has role=caption and this is the outcome when testing. And you are correct about await page.getByRole('caption').click(); working. As I stated in my initial post, navigating to just the role works but trying to use a name with that specific role does not whereas it does with role=button and others. There is no mention in the documentation that some roles will work with names and others won't so I assume there could be a bug. When using the "pick locator" option it will recommend page.getbytext(), indicating it does not recognize the role. If I change the role to a button then it recommends page.getbyrole(button, {name: [name]};. I did notice that Edge has an auto-complete in the inspector's element tab that shows you a list of role options. That list has the following: alert; application; article; banner; button; cell; checkbox; complementary; contentinfo; dialog; document; feed; figure; form; grid; gridcell; heading; img; list; listbox; listitem; main; navigation; region; row; rowgroup; search; switch; tab; table; tabpanel; textbox; timer. This means that the following may not be supported anymore? alertdialog; blockquote; caption; code; columnheader; combobox; definition; deletion; directory; emphasis; generic; group; insertion; link; log; marquee; math; meter; menu; menubar; menuitem; menuitemcheckbox; menuitemradio; none; note; option; paragraph; presentation; progressbar; radio; radiogroup; rowheader; scrollbar; searchbox; separator; slider; spinbutton; status; strong; subscript; tablist; superscript; term; time; toolbar; tooltip; tree; treegrid; treeitem

[dgozman]

Investigation notes. For the following snippet, there are various opinions for the roles/names.

<a id="link1" href="example.com" role="caption">NAME1</a>
<a id="link2" href="example.com" role="caption" aria-label="LABEL2">NAME2</a>
<a id="link3" href="example.com" role="term">NAME3</a>
<a id="link4" href="example.com" role="term" aria-label="LABEL4">NAME4</a>
<a id="link5" href="example.com" role="presentation">NAME5</a>
<a id="link6" href="example.com" role="presentation" aria-label="LABEL6">NAME6</a>

• Chromium: "NAME1", "LABEL2", "", "LABEL4", "NAME5" (role=link), "LABEL6" (role=link)
• Firefox: "", "LABEL2", "NAME3", "LABEL4", "NAME5" (role=link), "LABEL6" (role=link)
• WebKit: "", "LABEL2", "", "LABEL4", "NAME5" (role=link), "LABEL6" (role=link)
• ARIA spec: "", "LABEL2", "", "LABEL4", "" (role=presentation), "LABEL6" (role=link)

[kristojorg]

Role alert also seems affected by this. I can select an element with getByRole("alert") but am unable to select the element with getByRole("alert", { name: "<string child contents>" })

[MaruschkaScheepersW]

This issue seems to affect combobox and generic as well.

I can select the combobox just fine with this.portalSwitch = page.getByRole('combobox').first();

Tests of mine that previously worked are now failing where role is set to 'generic'

But as soon as I specify the name, it fails to select the element.

[dgozman]

@kristojorg @MaruschkaScheepersW Could you please file a new issue by filling in the "Bug Report" template and providing a repro? ARIA is full of tricky details, so we need the exact scenario to make sure there is a proper fix.

[jdgarvey]

@dgozman I also ran into this issue with the combobox, I submitted a new issue with a minimal reproduction here: https://github.com/microsoft/playwright/issues/31135.