Segmentation Fault in Lightpanda during Playwright CDP Connection

[adamdelezuch89]

Description

I am encountering a segmentation fault within the lightpanda browser when attempting to connect to it using Playwright's connect_over_cdp method. This occurs immediately after websocket connection is established successfully and when playwright tries to get browser version from cdp. This issue prevents me from using Playwright with lightpanda.

Steps to Reproduce

Set the LIGHTPANDA_PATH environment variable to the path of the lightpanda executable.

Run the following Python code:

from dotenv import load_dotenv

load_dotenv()


def test_connect_browser():
    import os
    from playwright.sync_api import sync_playwright
    import os
    import subprocess
    import time

    lightpanda_path = os.getenv("LIGHTPANDA_PATH", "")

    process = subprocess.Popen([
        lightpanda_path,
        "--host", "127.0.0.1",
        "--port", "9222",
        "--verbose",
    ])

    time.sleep(2)  # 2 sekundy powinny wystarczyć

    with sync_playwright() as p:
        ws_endpoint = "ws://127.0.0.1:9222"  # Uproszczony endpoint bez dodatkowej ścieżki
        browser = p.chromium.connect_over_cdp(ws_endpoint)
        
        page = browser.new_page()
        page.goto("https://mail.google.com/mail/u/0/#inbox")
        page.wait_for_load_state()

        input("Press the Enter key to close the browser...")

        browser.close()
        process.terminate()  # Dodajemy zamknięcie procesu


if __name__ == '__main__':
    test_connect_browser()

Error Logs

python tests/test_playwright.py 
debug(cli): Server opts: listening internally on /tmp/lightpanda...
info(websocket): starting blocking worker to listen on 127.0.0.1:9222
info(server): accepting new conn...
debug(websocket): (127.0.0.1:54548) connected
debug(handler): connected to Stream server
debug(websocket): (127.0.0.1:54548) connection successfully upgraded
info(server): client connected
debug(websocket): (127.0.0.1:54548) received text message
debug(cdp): Req > id 1, method browser.getVersion
debug(cdp): Res > id 1, sessionID null, result { }
Segmentation fault at address 0x4
/home/runner/work/browser/browser/vendor/websocket.zig/src/server/server.zig:1289:28: 0x1545fe8 in writeFrame (lightpanda)
/home/runner/work/browser/browser/vendor/websocket.zig/src/server/server.zig:1238:31: 0x154632d in send (lightpanda)
/home/runner/work/browser/browser/src/server.zig:278:29: 0x160b6f2 in wrapper (lightpanda)
/opt/hostedtoolcache/zig/0.13.0/x64/lib/std/debug.zig:0:9: 0x152d042 in run_for_ns (lightpanda)
/home/runner/work/browser/browser/src/server.zig:476:31: 0x152b238 in handle (lightpanda)
/opt/hostedtoolcache/zig/0.13.0/x64/lib/std/Thread.zig:429:13: 0x152e0ae in entryFn (lightpanda)
./nptl/pthread_create.c:442:8: 0x7e5b05294ac2 in start_thread (pthread_create.c)
../sysdeps/unix/sysv/linux/x86_64/clone3.S:81:0: 0x7e5b0532684f in ??? (../sysdeps/unix/sysv/linux/x86_64/clone3.S)
???:?:?: 0x0 in ??? (???)
debug(websocket): (127.0.0.1:54548) connection closed: error.Closed
error(handler): stream close error: error.BrokenPipe
Traceback (most recent call last):
  File "/home/temp/web-ui/tests/test_playwright.py", line 39, in <module>
    test_connect_browser()
  File "/home/temp/web-ui/tests/test_playwright.py", line 26, in test_connect_browser
    browser = p.chromium.connect_over_cdp(ws_endpoint)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/temp/web-ui/.venv/lib/python3.11/site-packages/playwright/sync_api/_generated.py", line 14826, in connect_over_cdp
    self._sync(
  File "/home/temp/web-ui/.venv/lib/python3.11/site-packages/playwright/_impl/_sync_base.py", line 115, in _sync
    return task.result()
           ^^^^^^^^^^^^^
  File "/home/temp/web-ui/.venv/lib/python3.11/site-packages/playwright/_impl/_browser_type.py", line 174, in connect_over_cdp
    response = await self._channel.send_return_as_dict("connectOverCDP", params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/temp/web-ui/.venv/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 67, in send_return_as_dict
    return await self._connection.wrap_api_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/temp/web-ui/.venv/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 528, in wrap_api_call
    raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
playwright._impl._errors.TimeoutError: BrowserType.connect_over_cdp: Timeout 30000ms exceeded.
Call log:
  - <ws connecting> ws://127.0.0.1:9222/
  -   - <ws connected> ws://127.0.0.1:9222/

Environment

Operating System: Ubuntu 22.04

Python Version: 3.12.7

Playwright Version: 1.49.1

Lightpanda Version: Nightly

[krichprollsch]

Hello @adamdelezuch89, Thanks for the report. We have issues with playwright's CDP support. I will try to reproduce yours in the next days.

[adamdelezuch89]

Hi, @krichprollsch. Something new about this issue?

[krichprollsch]

Hello @adamdelezuch89, Thanks for the reminder. Unfortunately I didn't work on Playwright compat. We are refactoring the CDP implementation, I hope I will take a look after.

[luisnquin]

I wanted to experiment with the browser + Playwright

import { chromium } from "playwright";

const browser = await chromium.connect("ws://localhost:9222");

But I've got the following logs:

connect: WebSocket error: ws://localhost:9222/ 400 Bad Request
Invalid connection header
Call log:
  - <ws connecting> ws://localhost:9222/
  - <ws unexpected response> ws://localhost:9222/ 400 Bad Request
Invalid connection header
  - <ws error> ws://localhost:9222/ error WebSocket was closed before the connection was established
  - <ws connect error> ws://localhost:9222/ WebSocket was closed before the connection was established
  - <ws disconnected> ws://localhost:9222/ code=1006 reason=

Version information

[krichprollsch]

@adamdelezuch89 @luisnquin we make progress with the support of Playwright recently. Can you retry your tests please?

I went to the Press the Enter key to close the browser... step with my python's test.

[luisnquin]

@krichprollsch I had the same error when trying to implement it in a new project I have.

[krichprollsch]

@luisnquin do you use the same script? It could be a later command which fails.

It seems you try to close the page/target once the error happens, so we can't see the root cause of the error.

[luisnquin]

@krichprollsch here's a reproducible example

It's from a freelance project so the source code shouldn't be public yet but here it is to help you: https://github.com/0xc000022070/gmaps-web-scraper.

After cloning the project apply this diff:

diff --git a/index.ts b/index.ts
index 6131550..544ef13 100644
--- a/index.ts
+++ b/index.ts
@@ -65,10 +65,7 @@ function parseDetails(businessName: string, raw: string): BusinessMeta {
 }
 
 async function getBusinessData(criteria: string) {
-  const browser = await playwright.chromium.launch({
-    args: ["--no-sandbox"],
-    headless: false,
-  });
+  const browser = await playwright.chromium.connect("ws://localhost:9222");
 
   const ctx = await browser.newContext();

Run the script with Bun and with the following API call you'll be able to see the error in the response:

$ curl 'http://localhost:3000/_x/Clínicas%20estéticas%20en%20Málaga'

[krichprollsch]

Hey @luisnquin, Really sorry for not answering earlier... Could you retry with this change for the connection:

const browser = await chromium.connectOverCDP({
    endpointURL: "ws://localhost:9222",
});