crawl4ai
crawl4ai copied to clipboard
fix cleanup warning when no process listening on debug port
Summary
The following warning is raised on Linux when using use_persistent_context=True without any existing process listening to the debugging port:
[BROWSER]. βΉ pre-launch cleanup failed: Command '[['lsof', '-t', '-i:9222']]'
Can be seen at line 2 in the Error logs of https://github.com/unclecode/crawl4ai/issues/1138#issue-3077595076
This is caused by lsof returning an error on empty search result, since no process is listening.
List of files changed and why
The fix changes crawl4ai.browser_manager.ManagedBrowser.start to pass -Q to lsof:
-Q ignore failed search terms. [β¦] lsof will return an error if any of the search results are empty. The -Q option will change this behavior so that lsof will instead return a successful exit code
How Has This Been Tested?
Current behavior when no process is listening:
https://github.com/unclecode/crawl4ai/blob/897e0173618d20fea5d8952ccdbcdad0febc0fee/crawl4ai/browser_manager.py#L217-L220
>>> subprocess.check_output(shlex.split("lsof -t -i:1234")).decode().strip().splitlines()
Traceback (most recent call last):
File "<python-input-5>", line 1, in <module>
subprocess.check_output(shlex.split("lsof -t -i:1234")).decode().strip().splitlines()
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.13/subprocess.py", line 472, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**kwargs).stdout
^^^^^^^^^
File "/usr/lib64/python3.13/subprocess.py", line 577, in run
raise CalledProcessError(retcode, process.args,
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['lsof', '-t', '-i:1234']' returned non-zero exit status 1.
New behavior
>>> subprocess.check_output(shlex.split("lsof -Q -t -i:1234")).decode().strip().splitlines()
[]
>>> subprocess.check_output(shlex.split("lsof -Q -t -i:9222")).decode().strip().splitlines()
['76965', '76973']
Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] I have added/updated unit tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
Summary by CodeRabbit
- Bug Fixes
- Improved reliability of browser startup cleanup on macOS/Linux systems by updating the process detection command.
Walkthrough
The update changes the command used to identify processes listening on a specific port during browser startup cleanup. Specifically, it adds the -Q flag to the lsof command on macOS/Linux, altering how process information is retrieved before termination. No other logic or public interfaces are modified.
Changes
| File(s) | Change Summary |
|---|---|
| crawl4ai/browser_manager.py | Updated lsof command to include -Q flag for process lookup. |
Poem
A bunny tweaks a single line,
For browsers neat and ports divine.
With-Qin tow, the search is new,
Fewer hiccups, smoother through.
Hopping on with gentle might,
The code is cleanerβoh, what delight!
π Recent review details
Configuration used: CodeRabbit UI Review profile: CHILL Plan: Pro
π₯ Commits
Reviewing files that changed from the base of the PR and between 897e0173618d20fea5d8952ccdbcdad0febc0fee and 8013ea8024bfba5db5365a431d076d6b10e786c6.
π Files selected for processing (1)
crawl4ai/browser_manager.py(1 hunks)
π Additional comments (1)
crawl4ai/browser_manager.py (1)
217-217: LGTM! Clean fix for the cleanup warning issue.The addition of the
-Qflag to thelsofcommand correctly addresses the reported warning when no process is listening on the debug port. This flag instructslsofto ignore failed search terms and return a successful exit code, which is exactly the desired behavior for cleanup logic.
β¨ Finishing Touches
- [ ] π Generate Docstrings
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
πͺ§ Tips
Chat
There are 3 ways to chat with CodeRabbit:
- Review comments: Directly reply to a review comment made by CodeRabbit. Example:
I pushed a fix in commit <commit_id>, please review it.Explain this complex logic.Open a follow-up GitHub issue for this discussion.
- Files and specific lines of code (under the "Files changed" tab): Tag
@coderabbitaiin a new review comment at the desired location with your query. Examples:@coderabbitai explain this code block.@coderabbitai modularize this function.
- PR comments: Tag
@coderabbitaiin a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:@coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.@coderabbitai read src/utils.ts and explain its main purpose.@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.@coderabbitai help me debug CodeRabbit configuration file.
Support
Need help? Create a ticket on our support page for assistance with any issues or questions.
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.
CodeRabbit Commands (Invoked using PR comments)
@coderabbitai pauseto pause the reviews on a PR.@coderabbitai resumeto resume the paused reviews.@coderabbitai reviewto trigger an incremental review. This is useful when automatic reviews are disabled for the repository.@coderabbitai full reviewto do a full review from scratch and review all the files again.@coderabbitai summaryto regenerate the summary of the PR.@coderabbitai generate docstringsto generate docstrings for this PR.@coderabbitai generate sequence diagramto generate a sequence diagram of the changes in this PR.@coderabbitai resolveresolve all the CodeRabbit review comments.@coderabbitai configurationto show the current CodeRabbit configuration for the repository.@coderabbitai helpto get help.
Other keywords and placeholders
- Add
@coderabbitai ignoreanywhere in the PR description to prevent this PR from being reviewed. - Add
@coderabbitai summaryto generate the high-level summary at a specific location in the PR description. - Add
@coderabbitaianywhere in the PR title to generate the title automatically.
CodeRabbit Configuration File (.coderabbit.yaml)
- You can programmatically configure CodeRabbit by adding a
.coderabbit.yamlfile to the root of your repository. - Please see the configuration documentation for more information.
- If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation:
# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json
Documentation and Community
- Visit our Documentation for detailed information on how to use CodeRabbit.
- Join our Discord Community to get help, request features, and share feedback.
- Follow us on X/Twitter for updates and announcements.