crawl4ai icon indicating copy to clipboard operation
crawl4ai copied to clipboard

fix cleanup warning when no process listening on debug port

Open lbeziaud opened this issue 6 months ago β€’ 1 comments

Summary

The following warning is raised on Linux when using use_persistent_context=True without any existing process listening to the debugging port:

[BROWSER]. β„Ή pre-launch cleanup failed: Command '[['lsof', '-t', '-i:9222']]'

Can be seen at line 2 in the Error logs of https://github.com/unclecode/crawl4ai/issues/1138#issue-3077595076

This is caused by lsof returning an error on empty search result, since no process is listening.

List of files changed and why

The fix changes crawl4ai.browser_manager.ManagedBrowser.start to pass -Q to lsof:

-Q ignore failed search terms. […] lsof will return an error if any of the search results are empty. The -Q option will change this behavior so that lsof will instead return a successful exit code

How Has This Been Tested?

Current behavior when no process is listening:

https://github.com/unclecode/crawl4ai/blob/897e0173618d20fea5d8952ccdbcdad0febc0fee/crawl4ai/browser_manager.py#L217-L220

>>> subprocess.check_output(shlex.split("lsof -t -i:1234")).decode().strip().splitlines()
Traceback (most recent call last):
  File "<python-input-5>", line 1, in <module>
    subprocess.check_output(shlex.split("lsof -t -i:1234")).decode().strip().splitlines()
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/subprocess.py", line 472, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               **kwargs).stdout
               ^^^^^^^^^
  File "/usr/lib64/python3.13/subprocess.py", line 577, in run
    raise CalledProcessError(retcode, process.args,
                             output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['lsof', '-t', '-i:1234']' returned non-zero exit status 1.

New behavior

>>> subprocess.check_output(shlex.split("lsof -Q -t -i:1234")).decode().strip().splitlines()
[]
>>> subprocess.check_output(shlex.split("lsof -Q -t -i:9222")).decode().strip().splitlines()
['76965', '76973']

Checklist:

  • [x] My code follows the style guidelines of this project
  • [x] I have performed a self-review of my own code
  • [ ] I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have added/updated unit tests that prove my fix is effective or that my feature works
  • [ ] New and existing unit tests pass locally with my changes

Summary by CodeRabbit

  • Bug Fixes
    • Improved reliability of browser startup cleanup on macOS/Linux systems by updating the process detection command.

lbeziaud avatar May 27 '25 20:05 lbeziaud

Walkthrough

The update changes the command used to identify processes listening on a specific port during browser startup cleanup. Specifically, it adds the -Q flag to the lsof command on macOS/Linux, altering how process information is retrieved before termination. No other logic or public interfaces are modified.

Changes

File(s) Change Summary
crawl4ai/browser_manager.py Updated lsof command to include -Q flag for process lookup.

Poem

A bunny tweaks a single line,
For browsers neat and ports divine.
With -Q in tow, the search is new,
Fewer hiccups, smoother through.
Hopping on with gentle might,
The code is cleanerβ€”oh, what delight!


πŸ“œ Recent review details

Configuration used: CodeRabbit UI Review profile: CHILL Plan: Pro

πŸ“₯ Commits

Reviewing files that changed from the base of the PR and between 897e0173618d20fea5d8952ccdbcdad0febc0fee and 8013ea8024bfba5db5365a431d076d6b10e786c6.

πŸ“’ Files selected for processing (1)
  • crawl4ai/browser_manager.py (1 hunks)
πŸ”‡ Additional comments (1)
crawl4ai/browser_manager.py (1)

217-217: LGTM! Clean fix for the cleanup warning issue.

The addition of the -Q flag to the lsof command correctly addresses the reported warning when no process is listening on the debug port. This flag instructs lsof to ignore failed search terms and return a successful exit code, which is exactly the desired behavior for cleanup logic.

✨ Finishing Touches
  • [ ] πŸ“ Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❀️ Share
πŸͺ§ Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

coderabbitai[bot] avatar May 27 '25 20:05 coderabbitai[bot]