Allowing Danswer to scrape information from the internet

[abfshaal]

Hello!

I would first like to say that Danswer is an amazing tool. It has a very seamless setup process!

I would like to request the addition of a feature for allowing searching on the internet. It can be done through agents, that would classify whether this request could be answered with rag approach, or to go online and do a search (could be google or tavily), then format then use the rag approach on the found resources (adding them as context).

I am happy to also contribute this myself if the authors approve it.