generative-ai-cdk-constructs icon indicating copy to clipboard operation
generative-ai-cdk-constructs copied to clipboard

bedrock: initial sync for S3DataSource attached to KnowledgeBase

Open jlosito opened this issue 1 year ago • 2 comments

Describe the feature

I would like the ability to run an initial sync of a Data source that is associated with a Knowledge base through a property.

Use Case

I have a use case where I have an S3 bucket with objects already in it. I'm using this library to provision the Knowledge Base and Data source via the KnowledgeBase and S3DataSource. When the stack completes, I have to go an start an ingestion job through a script, but much rather do this via a property.

Other constructs have similar functionality, but not exactly. For instance, the Bucket construct has an autoDeleteObjects property. It does this via a custom resource. I think something similar could be achieved for the Data sources via a custom resource that simply calls StartIngestionJob.

Proposed Solution

new bedrock.S3DataSource(this, "KnowledgeBaseDataSource", {
    knowledgeBase: myKnowledgeBase,
    bucket: myBucket,
    dataSourceName: myDataSourceName,
    initialSync: true,
});

Other Information

No response

Acknowledgements

  • [ ] I may be able to implement this feature request
  • [ ] This feature might incur a breaking change

jlosito avatar Apr 25 '24 15:04 jlosito

+1, Ingestion job is needed to sync with existing s3 contents.

1vinodsingh1 avatar Jun 02 '24 19:06 1vinodsingh1

+1, Ingestion job is needed to sync with existing s3 contents.

vaughngit avatar Oct 04 '24 18:10 vaughngit