cloudflare-vector-search

cloudflare-vector-search copied to clipboard

Haven't yet taking a deeper look, but given the existence of Cloudflare Queues, which support queueing up jobs, and this library batching of large documents, could be a great addition to add queues to the wrangler.toml file.

It will involve a bit more from the user, namely creating the queues (can be automated with a bash script or user instructions)

wrangler queues create batch_vector_storage

But it should in theory make this be able to process huge documents without additional extra pieces

Interesting idea!