weaviate-python-client icon indicating copy to clipboard operation
weaviate-python-client copied to clipboard

Batch import doesn't apply rate limit when using async indexing

Open rlmanrique opened this issue 1 year ago • 0 comments

Using the Python client in its version 4.10.4

The function rate_limit is not working if Weaviate is configured in async indexing mode:

    with open(dataset_path, "r") as f:
        collection.batch.rate_limit(100)
        with collection.batch.dynamic() as batch:
            for raw_json_email in f:
                email = json.loads(raw_json_email)
                batch.add_object(
                    properties=email["data"],
                    uuid=email["id"],
                )

The batches have 1000 objects, no mater the rate configured.

rlmanrique avatar Feb 06 '25 15:02 rlmanrique