erlexec icon indicating copy to clipboard operation
erlexec copied to clipboard

add a helper method that generates files from an s3 bucket or s3 uri that returns a generator of Documents like

Open JoanFM opened this issue 2 years ago • 0 comments

creating a storage backend based on S3 does not make much sense because it's not a vector database (therefore no ANN support or so). However, we can add a helper method that generates files from an s3 bucket or s3 uri that returns a generator of Documents like: https://docarray.jina.ai/api/docarray.document.generators/?highlight=from_files#docarray.document.generators.from_files

Later you can index these docs in a proper storage backend in docarray

Originally posted by @alaeddine-13 in https://github.com/jina-ai/docarray/issues/377#issuecomment-1150801340

JoanFM avatar Aug 04 '22 08:08 JoanFM