Rod Panics when receiving status 408 from Browserless

[jackHedaya]

Rod Version: v0.116.2

The code to demonstrate your question

• Run: docker run -p 9222:3000 ghcr.io/browserless/chromium

func TestRod(t *testing.T) {
	testChannel := make(chan string, 100)

	wg := &sync.WaitGroup{}

	go func() {
		for i := 0; i < 10_000; i++ {
			testChannel <- "https://example.com"
		}

		close(testChannel)
	}()

	for i := 0; i < 10; i++ {
		wg.Add(1)
		go worker(context.Background(), testChannel, wg)
	}

	wg.Wait()

}

func worker(ctx context.Context, sites <-chan string, wg *sync.WaitGroup) {
	defer wg.Done()

	defer func() {
		if r := recover(); r != nil {
			fmt.Printf("recovered from panic: %v\n", r)
		}
	}()

	for {
		select {
		case site := <-sites:
			browser, err := createNewBrowser(ctx)

			if err != nil {
				panic(fmt.Errorf("failed to create new browser: %w", err))
			}

			page := browser.MustPage(site)

			time.Sleep(40 * time.Second)

			page.MustWaitLoad()

			fmt.Println(page.MustElement("body").MustText())

			page.Close()
			browser.Close()

		case <-ctx.Done():
			return
		}
	}
}

func createNewBrowser(ctx context.Context) (_r *rod.Browser, e error) {
	defer func() {
		if r := recover(); r != nil {
			e = fmt.Errorf("createNewBrowser: recovered from panic: %v", r)
		}
	}()

	serviceUrl := "ws://localhost:9222"
	u, err := launcher.ResolveURL(serviceUrl)

	if err != nil {
		return nil, fmt.Errorf("failed to resolve url: %w", err)
	}

	browser := rod.New().ControlURL(u)

	err = browser.Connect()

	if err != nil {
		return nil, fmt.Errorf("failed to connect to browser: %w", err)
	}

	return browser, nil
}

What you got

Panic (invalid character 'T' looking for beginning of value) due to error:

TP/1.1 408 Request Timeout
Content-Type: text/plain; charset=UTF-8
Content-Encodin

What you expect to see

No panic.

What have you tried to solve the question

Solution is not straightforward as the panicking line is inside of a non-returning Go routine: https://github.com/go-rod/rod/blob/main/lib/cdp/client.go#L148

Reference in Browserless: https://github.com/browserless/browserless/issues/4418

[ViRb3]

I started experiencing the same issue, making it impossible to use Rod. Is there any workaround?

[williamoliveira]

I found the issue, browserless is closing the WS connection after a default timeout configuration of 30 seconds, and its sending a HTTP response 408 that the Rod is not handling as it expects the CDPs connections should keep open forever.

To fix this I just set TIMEOUT=0 on browserless env so the connections will keep open indefinitely.

An improvement on Rod side would be to handle CDP WS connections unexpectedly closing in a more graceful manner