[Feature]: add snapshotForAI support

[wangzhe0912]

🚀 Feature Request

add snapshotForAI support. base Python version inplement PlayWright MCP Server。

Example

No response

Motivation

base Python version inplement PlayWright MCP Server。

[Fly-Playgroud]

Now many users use AI driving browser by playwright, add snapshotForAI in python version is important. I support this feature.

[neoncube2]

What would snapshotForAI do?

[wangzhe0912]

https://github.com/microsoft/playwright/blob/bd5a23f88f3c54b6fd15ff1cde0693babfc86285/packages/protocol/src/channels.d.ts#L2095 The JS version of Playwright provides a snapshotForAI method, which converts a Page into a snapshot format. In this format, each page element is assigned a ref-id, enabling models to understand the page and perform actions based on the ref-id.

Currently, the Python version does not expose this method and only supports a snapshot method based on a specified locator. This method differs from snapshotForAI, as some elements (e.g., text elements) are not assigned a ref-id, making it unsuitable for AI to identify elements and perform intelligent operations.

[latifboubyan]

Any updates on this? I have a use case not satisfied by the MCP so I'd like to use the python library directly, but I want to make use of snapshotForAI.

[changshuo08]

@latifboubyan Could you give me some examples for mcp

[wangzhe0912]

Or exposing the forAI parameter in the ariaSnapshot method in the Node version would also work.

Attached is the ariaSnapshot definition from Node version 1.53:

ariaSnapshot:
  title: Aria snapshot
  parameters:
    selector: string
    forAI: boolean?
    timeout: number
  returns:
    snapshot: string
  flags:
    snapshot: true

https://github.com/microsoft/playwright/blob/main/packages/protocol/src/protocol.yml#L2108

[zia1138]

I'd like to drive playwright using custom tools in my LLM agent. Being able to use snapshotForAI directly would be awesome instead of through playwright-mcp.

[wangzhe0912]

The Python Playwright 1.53.0 has been released. Can the for_ai parameter be added to the aria_snapshot method now?

[Fly-Playgroud]

The Python Playwright 1.53.0 has been released. Can the for_ai parameter be added to the aria_snapshot method now?

I have hacked this in Playwright-Python 1.53.0. It's ran prefectly. But I still hope offical support this.

[HitSakhavala]

The Python Playwright 1.53.0 has been released. Can the for_ai parameter be added to the aria_snapshot method now?

I have hacked this in Playwright-Python 1.53.0. It's ran prefectly. But I still hope offical support this.

@Fly-Playground, how did you achieve this? Can you help me set it up the same?

[Gby56]

@Fly-Playgroud @HitSakhavala I assume the hack is via the channel, to send the aisnapshot command directly ?

[argszero]

await page._impl_obj._channel.send('snapshotForAI', None, {'timeout': 5000})

[idiotWu]

For those who want to get the aria snapshot for a locator with refs (which is not possible via locator.aria_snapshot()):

await locator._impl_obj._frame._channel.send(
    "ariaSnapshot",
    locator._impl_obj._frame._timeout,
    {
        "selector": locator._impl_obj._selector,
        "timeout": 5000,
        "forAI": True,  # this param does the magic
    },
)

[zia1138]

Sharing another example in case it is helpful for others:

from playwright.async_api import async_playwright

URL = "https://www.businesswire.com/news/home/20250829814571/en/Merck-Provides-New-Results-for-VERQUVO-vericiguat-in-Patients-with-Chronic-Heart-Failure-and-Reduced-Ejection-Fraction"
async def grab_snapshot(url: str):
    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=False)
        page = await browser.new_page()
        await page.goto(url, timeout=60000)

        snapshot = await page._impl_obj._channel.send(
            "snapshotForAI", None, {"timeout": 5000}
        )

        await browser.close()
        return snapshot
    
snapshot = await grab_snapshot(URL)

[baijiu-in-my-cup]

Sharing another example in case it is helpful for others:分享另一个例子，以防对其他人有帮助：

from playwright.async_api import async_playwright

URL = "https://www.businesswire.com/news/home/20250829814571/en/Merck-Provides-New-Results-for-VERQUVO-vericiguat-in-Patients-with-Chronic-Heart-Failure-and-Reduced-Ejection-Fraction" async def grab_snapshot(url: str): async with async_playwright() as p: browser = await p.chromium.launch(headless=False) page = await browser.new_page() await page.goto(url, timeout=60000)

    snapshot = await page._impl_obj._channel.send(
        "snapshotForAI", None, {"timeout": 5000}
    )

    await browser.close()
    return snapshot

snapshot = await grab_snapshot(URL)

Hi, I used your method and got a ref, but I don’t see any ref attribute on the actual page. How exactly can I use this ref to locate the element?

[zia1138]

@baijiu-in-my-cup Not 100 sure. I think you need to keep the session open and use the refs in the output snapshot. I think that's how the playwright MCP server does it, but I didn't explore this extensively.

[baijiu-in-my-cup]

@zia1138 Thanks♪(･ω･)ﾉ I'm all set now ! It turns out I made a mistake earlier,I thought it was a CSS-style selector, but it's actually a register selector. So I need to do is use locator(f"aria-ref={ref}"). Thank you to all the developers,amazing!

[Gby56]

Hello @mxschmitt 👋 I was thinking back on this ticket, I'm waiting on this to be officially supported in order to update my version of PW, is this planned any time soon?

If the complexity isn't too bad I could try to make a PR for it !