langchain
langchain copied to clipboard
Document type and github loader
This is a proof of concept, I did a quick dump of how I thought this should work.
Add LangChain.Document module, which provides a structured way to handle documents with content, metadata, and type attributes.
Add LangChain.Document.Loaders.Github module to support fetching issues from a specified repository, converting them into Document structs.
gh = LangChain.Document.Loaders.Github.new!()
docs = LangChain.Document.Loaders.Github.load(gh, %{type: :issue, repo: "brainlid/langchain"}
Nice @matthusby! You describe this as a PoC. Is there anything else you think it needs or is lacking?
Would you be willing to write tests for the lib/document/loaders/github.ex file?
I called it PoC as its only doing issues, I thought I would add some more to it. But I have gotten busy so this can go as is and I will add more to it when I expand it. Added some tests 👍