kedro-plugins icon indicating copy to clipboard operation
kedro-plugins copied to clipboard

feat(datasets): Add limited `langchain` support for Anthropic, Cohere, and OpenAI models

Open ianwhale opened this issue 7 months ago • 3 comments

Description

Adds limited support for langchain models.

This PR is a rough starting point for loading langchain API-based models.

The big issue here is langchain's model catalog. See the list here (just for chat models).

There's no way anyone could implement and maintain all of these.

Even if that was desirable, we can see from the CohereDataset example that there are going to be lots of details along the way that will make this task difficult.

Would love to see what the team thinks and if this is worth pushing forward!

Development notes

Adds four datasets for interacting with langchain models.

Checklist

  • [x] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • [ ] Updated the documentation to reflect the code changes
  • [ ] Added a description of this change in the relevant RELEASE.md file
  • [ ] Added tests to cover my changes

ianwhale avatar Nov 16 '23 22:11 ianwhale