Ability to set vector IDs through PineconeStore.fromDocuments

[stephenasuncionDEV]

It may be worth considering adding the ability to set vector IDs through PineconeStore.fromDocuments. Currently, it is impossible to fetch vectors from Pinecone's dashboard without knowing the ID. Upon inspection, it appears that if the IDs parameter is undefined, the system generates an ID from UUIDv4 (https://github.com/hwchase17/langchainjs/blob/main/langchain/src/vectorstores/pinecone.ts#L52). However, as it stands, there is no way to pass an ID array from fromDocuments to addDocuments, so it will always be undefined for the fromDocuments function.

A simple fix would be to add an "ids" field in dbConfig:PineconeLibArgs and pass it to addDocuments.

[moghwan]

same issue here with Supabase, we can specify a table name with .fromText() but not with .fromDocuments()

[evannorstrand-mp]

Bumping this, as it should be a high priority

[dosubot[bot]]

Hi, @stephenasuncionDEV! I'm here to help the LangChain team manage their backlog and I wanted to let you know that we are marking this issue as stale.

From what I understand, you raised an issue suggesting the addition of the ability to set vector IDs through PineconeStore.fromDocuments so that vectors can be fetched from Pinecone's dashboard without knowing the ID. Another user, moghwan, mentioned a similar issue with Supabase. evannorstrand-mp also bumped the issue, highlighting its high priority.

Before we proceed, we would like to confirm if this issue is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on this issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your contribution and we look forward to hearing from you soon!

[8times4]

Same issue for Milvus, only a few params can be set via MilvusLibArgs. I.e autoId and others like:

    indexCreateParams: {
        index_type: string;
        metric_type: string;
        params: string;
    }; 

can't be.

Thanks!

[dosubot[bot]]

@jacoblee93 Could you please help @8times4 with this issue? They are experiencing a similar problem with Milvus and have provided additional context in their comment. Thank you!

[dosubot[bot]]

Hi, @stephenasuncionDEV! I'm Dosu, and I'm here to help the langchainjs team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue is about adding the ability to set vector IDs through PineconeStore.fromDocuments in order to fetch vectors from Pinecone's dashboard without knowing the ID. There have been some discussions in the comments about similar issues with Supabase and Milvus, where additional parameters were requested to be set.

However, it seems that the issue is still unresolved. So, I wanted to ask you if this issue is still relevant to the latest version of the langchainjs repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the langchainjs repository! Let us know if you have any further questions or concerns.

[mowliv]

Ug. Still a problem AFAIK. Frustrating not to be able to set ID's with fromDocuments. Is there a workaround?

See this Pinecone RAG doc for justification.

[jacoblee93]

Yeah we should fix this. Will take a look, thanks for bumping

[mowliv]

Thanks very much!

On Wed, Jul 3, 2024 at 8:46 PM Jacob Lee @.***> wrote:

Yeah we should fix this. Will take a look, thanks for bumping

— Reply to this email directly, view it on GitHub https://github.com/langchain-ai/langchainjs/issues/571#issuecomment-2208068626, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDDXTOECB4OBM3HNJ7AK5DZKTAPVAVCNFSM6AAAAAAWP6MKSWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBYGA3DQNRSGY . You are receiving this because you commented.Message ID: @.***>

[mowliv]

As an FYI to others, it looks like PineconeStore.fromExistingIndex() is a workaround. Working for me.