support for pgvector vector store

[lujin1]

Self Checks

• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to file this report (我已阅读并同意 Language Policy).

1. Is this request related to a challenge you're experiencing?

cannot use existing pgvector

2. Describe the feature you'd like to see

support for pgvector vector store

3. How will this feature improve your workflow or experience?

use existing pgvector

4. Additional context or comments

No response

5. Can you help us with this feature?

• [ ] I am interested in contributing to this feature.

[dosubot[bot]]

🤖

Hello @lujin1! 😊 I'm Dosu, a friendly bot here to lend a hand while we wait for a human maintainer. I can help with resolving issues, answering questions, and guiding you through the contribution process. If you need anything, don't hesitate to ask!

Thank you for your feature request. Currently, dify supports three types of vector stores: "weaviate", "qdrant", and "milvus". This is determined by the VECTOR_STORE configuration value in the VectorIndex class.

To add support for pgvector, a new class (e.g., PgVectorIndex) that inherits from BaseVectorIndex and implements the required methods would need to be created. Then, a new condition in the _init_vector_index method would be added to handle the case where VECTOR_STORE is set to "pgvector".

Here is a simplified example of how this might look:

elif vector_type == "pgvector":
    from core.index.vector_index.pgvector_vector_index import PgVectorConfig, PgVectorIndex

    return PgVectorIndex(
        dataset=dataset,
        config=PgVectorConfig(
            host=config.get('PGVECTOR_HOST'),
            port=config.get('PGVECTOR_PORT'),
            user=config.get('PGVECTOR_USER'),
            password=config.get('PGVECTOR_PASSWORD'),
            secure=config.get('PGVECTOR_SECURE'),
        ),
        embeddings=embeddings
    )

Please note that the PgVectorIndex and PgVectorConfig classes would need to be implemented, and the actual configuration parameters might be different depending on the requirements of pgvector.

You can find the relevant code in the api/core/index/vector_index/vector_index.py file.

However, as I am an AI developed by OpenAI and do not have the ability to make changes to the repository, I would recommend reaching out to the repository maintainers or contributors to discuss this feature request further. They may be able to provide more specific guidance or even implement this feature.

I hope this information is helpful, and please let me know if you have any other questions.

Sources

• api/core/index/vector_index/vector_index.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[Yash-1511]

Hello @crazywoola ,

I am writing to express my interest in contributing to the development of support for the pgvector vector store in the project.

I believe that enhancing and adding support for the pgvector vector store will greatly improve my workflow and experience with the project. I have already conducted thorough research, including searching for existing issues (including closed ones), and I have confirmed that I am using English to file this report, in line with the Language Policy.

Thank you for considering my request, and I look forward to contributing to the project's success.

[crazywoola]

@Yash-1511 Thank you very much for your interest and enthusiasm in contributing to the project. Your suggestion to enhance support for pgvector vector storage is intriguing and could have a significant impact on the project's development.

Please feel free to share any specific ideas or implementation plans you have. We are always open to new ideas and welcome new contributors, and your support will undoubtedly enhance the overall quality and user experience of the project. Also, if you need any assistance or would like to discuss your ideas further, both the members of our community and I are more than willing to help.

Thank you again for your support and contribution!

You can contact me through email [email protected] or find pinkbanana in discord channel.

[github-actions[bot]]

Close due to it's no longer active, if you have any questions, you can reopen it.

[ThreeFish-AI]

Is there any progress on pgvector storage support?