pgsync
pgsync copied to clipboard
getting 429 while sending bulk request
PGSync version: 2.5.1
Postgres version: 12
Elasticsearch version: latest
Redis version: latest
Python version: 3.10
Problem Description: While updating 9000 row in db some time it post to open search but some time it through an exception of bulk request.
we have small instance. these parameter are set
ELASTICSEARCH_CHUNK_SIZE=1000 ELASTICSEARCH_MAX_CHUNK_BYTES=3242880 ELASTICSEARCH_MAX_RETRIES=14 ELASTICSEARCH_QUEUE_SIZE=1 ELASTICSEARCH_STREAMING_BULK=True ELASTICSEARCH_THREAD_COUNT=1 ELASTICSEARCH_TIMEOUT=320 QUERY_CHUNK_SIZE=500
Error Message (if any):
ERROR:pgsync.search_client: Exception TransportError(429, '429 Too Many Requests /_bulk')
data1-pgsync-1 | Traceback (most recent call last):