Marco Perini comments

Results 16 comments of


                                            Marco Perini

Change proxy rotation function

Thanks @DiTo97, I have just merged your PR #211 and added it to the documentation

pip install scrapegraphai for python3.12

Hey there are some dependencies which are not working for python3.12 so for now we have limited the range to python>=3.9,

Scraping n levels deep

Hei @rawmean, we will add it in the to-do list for feature requests! It would be interesting to create a new graph for this and maybe calling it `CrawlerGraph` or...

json.decoder.JSONDecodeError: Expecting value: line 10 column 5 (char 743)

Hei @david-strejc try set the "format":"json" in the llm dict and see if you still get the error. Sometimes with local models is needed

NotImplemented error while running in windows.

Hey try SearchGraph and see if you still have the problem. If no, then it is a asyncio problem in SmartScraper and will fix

How to custom base_url for OpenAI request?

@thangnqs yes the openai class verify first if the apikey is correct in the openai endpoints and then take the model from your url, the problem is if it requires...

insufficient_quota

Thanks for the feedback! We will add it, also it is possible to integrate a retry mechanism using [tenacity](https://tenacity.readthedocs.io/en/latest/index.html)

asyncio.run() cannot be called from a running event loop

Hey there, yes `Playwright` uses `asyncio` under the hood so probably you are trying to run an asyncio routine inside another one (your async crawl method). Right now the .rnu...

RuntimeError: asyncio.run() cannot be called from a running event loop

Duplicate of https://github.com/VinciGit00/Scrapegraph-ai/issues/179

Feature Request: Utilize Vision API for scraping

Hey @mikewolfd right now we have only one ImageToText node which uses gpt-4 vision [here](https://github.com/VinciGit00/Scrapegraph-ai/blob/main/scrapegraphai/nodes/image_to_text_node.py). It takes a URL of the image as input. We are planning to implement other...