langchain icon indicating copy to clipboard operation
langchain copied to clipboard

Document type and github loader

Open matthusby opened this issue 1 year ago • 2 comments

This is a proof of concept, I did a quick dump of how I thought this should work.

Add LangChain.Document module, which provides a structured way to handle documents with content, metadata, and type attributes.

Add LangChain.Document.Loaders.Github module to support fetching issues from a specified repository, converting them into Document structs.

gh = LangChain.Document.Loaders.Github.new!()
docs = LangChain.Document.Loaders.Github.load(gh, %{type: :issue, repo: "brainlid/langchain"}

matthusby avatar Aug 25 '24 20:08 matthusby

Nice @matthusby! You describe this as a PoC. Is there anything else you think it needs or is lacking?

Would you be willing to write tests for the lib/document/loaders/github.ex file?

brainlid avatar Nov 22 '24 04:11 brainlid

I called it PoC as its only doing issues, I thought I would add some more to it. But I have gotten busy so this can go as is and I will add more to it when I expand it. Added some tests 👍

matthusby avatar Jan 11 '25 00:01 matthusby