crawl4ai icon indicating copy to clipboard operation
crawl4ai copied to clipboard

Langchain - Document loader basic functionality i.e `lazy load` implemented Issue #77

Open aravindkarnam opened this issue 1 year ago • 2 comments

aravindkarnam avatar Sep 04 '24 04:09 aravindkarnam

@aravindkarnam, could we have LangChain dependencies as optional extras? LangChain is quite a complex and heavy library, so it would be better to have a core crawl service and then add extra dependencies if needed, rather than binding them from the start.

simjak avatar Oct 03 '24 06:10 simjak

@simjak, Agreed. The long term plan is to create a PR in langchain community repository for the crawl4ai langchain integration. We only started this here to get a first version out.

I'll add langchain dependancies as an extra here in the setup.py file. Sounds good @unclecode?

aravindkarnam avatar Oct 03 '24 16:10 aravindkarnam

@aravindkarnam Proceed, it totally make sense

unclecode avatar Nov 20 '24 10:11 unclecode