[Feature Request]Ability to connect to existing Knowledge Base

[vishallakhotia]

Describe the solution you'd like

As a user i already have an existing Knowledge Base(KB). I want to use this solution with my already existing KB

Why the solution needed

Customers might have KB on AWS already using certain embeddings model. If the embedding model match this solution, it would be helpful to integrate an existing KB without trying to re-upload all the files

Additional context

Add any other context or screenshots about the feature request here.

Implementation feasibility

Are you willing to discuss the solution with us, decide on the approach, and assist with the implementation?

• [X] Yes
• [ ] No

[statefb]

without trying to re-upload all the files

We're developing KB support. We'll add feature to import existing s3 bucket, but will not implement to import existing KB. Do you think this spec works for you?

[vishallakhotia]

importing existing S3 buckets is also very useful. How would the data be synced into the vector db ?

[statefb]

@vishallakhotia vector db will be depricated. instead of that, we use opensearch. We'd be appreciated if you could take a look at the issue above.

[Peter-Lankton]

without trying to re-upload all the files

We're developing KB support. We'll add feature to import existing s3 bucket, but will not implement to import existing KB. Do you think this spec works for you?

Yes, I'm starting to use the project, but might want to hold off until the KB is implemented. Is there an estimate on when it might be done (no rush just curious).

In the meanwhile, if I were to deploy the project with existing vectorDB arch; pgVector, then would the switch to KB be straightforward or would it require a redeploy?

I'm ok with needing a redeploy, just curious about the idea space for adding the new feature while handling the existence of previous.

[statefb]

Is there an estimate on when it might be done (no rush just curious).

ETA will be by the end of August.

would it require a redeploy?

At the initial release, KB will be a beta feature. This means we do not recommend using both pgvector and KB in the same application. Later on, it will be updated to v2, at which point pgvector will no longer be supported (if you wish to continue using it, deploy v1). We will not provide a straightforward procedure for moving to v2, which will require significant effort.

[Peter-Lankton]

Hey @statefb thank you for the reply. I'll wait until the end of august then and be a beta tester :). Awesome project by the way keep up the great work!

[github-actions[bot]]

This issue has been labeled as "stale" due to no response by the reporter within 1 month (and 14 days after last commented by someone). And it will be closed automatically 14 days later if not responded.