pr-agent icon indicating copy to clipboard operation
pr-agent copied to clipboard

Integration with other vectorDbs [enhancement]

Open PrashantDixit0 opened this issue 1 year ago • 16 comments
trafficstars

@mrT23 @okotek
Are you open to Integrate with other VectorDbs for giving flexibility to user to use PR-agent with any vectorDb of their choice?

I can work on integrating with ChromaDB and LanceDB for scalibility. Please let me know, What are your thoughts on this.

PrashantDixit0 avatar Dec 02 '23 06:12 PrashantDixit0

/similar_issue

mrT23 avatar Dec 02 '23 06:12 mrT23

I want to add it as an enhancement. Are you open to merge PR, If I create one for integrating other vectorDBs.

PrashantDixit0 avatar Dec 02 '23 06:12 PrashantDixit0

I want to add it as an enhancement. Are you open to merge PR, If I create one for integrating other vectorDBs.

Are you relating to the usage of the vectorDB in the /similar_issue command/feature? Or do you have a new feature/command in mind?

coditamar avatar Dec 02 '23 11:12 coditamar

I want to add LanceDB and ChromeDB also as a vectorDB with Pinecone.

Results from /similiar_issues are not what I have in my mind

PrashantDixit0 avatar Dec 02 '23 12:12 PrashantDixit0

I want to add LanceDB and ChromeDB also as a vectorDB with Pinecone.

Results from /similiar_issues are not what I have in my mind

Can you please clarify how will these vectorDB used?

coditamar avatar Dec 02 '23 12:12 coditamar

Same as Pinecone, they'll used for semantic searching and filtering similar PRs and Issues

PrashantDixit0 avatar Dec 02 '23 13:12 PrashantDixit0

@PrashantDixit0 Thanks for the suggestion.

We are open to adding another vectorDb provider. I would recommend focusing on a single one at the first stage.

i just want to emphasize this is not an easy task. The main challenge here is not actually doing the semantic search, but seamlessly managing the database. You need to decide, each time a user comments /similar_issue, if to build a fresh new database, or just update the existing one, and with what. This should be done efficiently and gracefully, and requires deep understanding of how the vectorDb is managed and can be handled.

mrT23 avatar Dec 02 '23 13:12 mrT23

@mrT23 Yes agree with you, I'll work on it and raise a PR soon.

Thank you for considering my suggestion ☺️

PrashantDixit0 avatar Dec 02 '23 14:12 PrashantDixit0

Sure. Feel free to ask questions and consult

mrT23 avatar Dec 02 '23 14:12 mrT23

Yes I'll update the progress on it with you

PrashantDixit0 avatar Dec 02 '23 14:12 PrashantDixit0

@mrT23 I have completed the Integration with LanceDB #523 . Please review it once

PrashantDixit0 avatar Dec 12 '23 11:12 PrashantDixit0

Similar Issues


mrT23 avatar Dec 14 '23 06:12 mrT23

@mrT23 I have created an updated PR #548 and kept all the comments you gave previously. Please review it once again

PrashantDixit0 avatar Dec 24 '23 19:12 PrashantDixit0