<Community>: Add Initial Support for TiDB Vector Store

[IANTHEREAL]

This pull request introduces initial support for the TiDB vector store. The current version is basic, laying the foundation for the vector store integration. While this implementation provides the essential features, we plan to expand and improve the TiDB vector store support with additional enhancements in future updates.

Upcoming Enhancements:

• Support for Vector Index Creation: To enhance the efficiency and performance of the vector store.
• Support for max marginal relevance search.
• Customized Table Structure Support: Recognizing the need for flexibility, we plan for more tailored and efficient data store solutions.

Simple use case exmaple

from typing import List, Tuple
from langchain.docstore.document import Document
from langchain_community.vectorstores import TiDBVector
from langchain_openai import OpenAIEmbeddings

db = TiDBVector.from_texts(
    embedding=embeddings,
    texts=['Andrew like eating oranges', 'Alexandra is from England', 'Ketanji Brown Jackson is a judge'],
    vectorstore_name="tidb_vector_langchain",
    connection_string=tidb_connection_url,
    distance_strategy="cosine",
)

query = "Can you tell me about Alexandra?"
docs_with_score: List[Tuple[Document, float]] = db.similarity_search_with_score(query)
for doc, score in docs_with_score:
    print("-" * 80)
    print("Score: ", score)
    print(doc.page_content)
    print("-" * 80)

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Mar 7, 2024 4:17am

[baskaryan]

@IANTHEREAL feel free to ping when the PR is ready for review

[IANTHEREAL]

@baskaryan The TiDB Vector feature is expected to release a preview version around the end of January. I will make the final adjustments then and invite you to review it 😊

[IANTHEREAL]

@hwchase17 @baskaryan

Quick update on the TiDB Vector Store PR:

• Development Complete: The core functionality of TiDB Vector Store is developed and now in internal testing.
• Staging in February: A stable staging environment will be ready by February for thorough review.
• Launch & Documentation in March: Planning for a March release with comprehensive documentation, including Langchain + TiDB Vector use cases.

Feel free to reach out with any questions or concerns. Excited to see Langchain and TiDB Vector Store in action!

[vercel[bot]]

Deployment failed with the following error:

The provided GitHub repository does not contain the requested branch or commit reference. Please ensure the repository is not empty.

[IANTHEREAL]

Thank you for your patience with this long-standing PR. I'm excited to tell that the TiDB Vector feature is now in the release pipeline, with most tests completed. We're targeting a release in the last two weeks of March.

And. the feature has been deployed in our staging environment for this integration review with LangChain. I'll share access credentials with reviewers shortly. Your review would be greatly appreciated, thanks in advance! @baskaryan @hwchase17

[IANTHEREAL]

Recently, there have been more and more code conflicts. If possible, please help review this PR. @baskaryan @hwchase17

And I've DM'd the access method for the tidb vector staging environment to @baskaryan, hope it's useful for this PR reviewing

[AV25242]

This is great @baskaryan can you please assist ?

[IANTHEREAL]

Thanks for reviewing @baskaryan @AV25242