use different service than browseless

[PiotrEsse]

Its possible to use different service than browseless as they drop free tier now (I cant find any information about free tier now and I dont want to be charged 200usd just for testing tools)?

[tblock-zz]

I like the project really much. Is it possible to use python libraries and not using commercial services like browserless, serper and aec-api.io? It would be fantastic if this would all run without external commercial services and with local LLMs.

[scottiesan]

you guys can try droping this...:

from bs4 import BeautifulSoup import requests

class BrowserTools(): @tool("Scrape website content") def scrape_and_summarize_website(website): """Useful to scrape and summarize a website content""" response = requests.get(website) soup = BeautifulSoup(response.text, 'html.parser') elements = partition_html(text=soup.prettify()) content = "\n\n".join([str(el) for el in elements]) content = [content[i:i + 8000] for i in range(0, len(content), 8000)] summaries = [] for chunk in content: agent = Agent( role='Principal Researcher', goal= 'Do amazing research and summaries based on the content you are working with', backstory= "You're a Principal Researcher at a big company and you need to do research about a given topic.", allow_delegation=False) task = Task( agent=agent, description= f'Analyze and summarize the content below, make sure to include the most relevant information in the summary, return only the summary nothing else.\n\nCONTENT\n----------\n{chunk}' ) summary = task.execute() summaries.append(summary) return "\n\n".join(summaries)

[greysonlalonde]

The answer @scottiesan provided is likely as close to "free" as you'll get without a complicated setup, albeit likely to hit snags at some point related to web scraping without an api key. This is not likely something that is going to change anytime soon, closing this