mindsdb_python_sdk icon indicating copy to clipboard operation
mindsdb_python_sdk copied to clipboard

Published 20 hours ago • verified publisher icon mindsdb

Add Support for Website Data Source for Minds

Open tmichaeldb opened this issue 6 months ago • 1 comments

This PR adds a new WebConfig subclass of DataSourceConfig. Specifically:

  • Updated agent.add_webpages to support crawl_depth and regex based URL filters
  • Added WebConfig class that configures web crawling a list of URLs for Minds

Part of ML-96

tmichaeldb avatar Aug 15 '24 18:08 tmichaeldb