[BUG] "ThrottlingException: Too many requests" when embedding on Amazon Bedrock

[frankyw]

Describe the bug When trying to upsert data and created embeddings via Amazon, the socket limit is reached and the AWS API throws an error (with a 4MB file). A smaller file with less records is successful.

2024-04-01 20:37:52 [INFO]: ⬆️ POST /api/v1/vector/internal-upsert/8bccc61b-3401-445b-9b8f-431f2c34a15b
@smithy/node-http-handler:WARN socket usage at capacity=50 and 1330 additional requests are enqueued. See https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/node-configuring-maxsockets.html or increase socketAcquisitionWarningTimeout=(millis) in the NodeHttpHandler config.
2024-04-01 20:38:00 [ERROR]: ThrottlingException: Too many requests, please wait before trying again. You have sent too many requests.  Wait before trying again.
Error: ThrottlingException: Too many requests, please wait before trying again. You have sent too many requests.  Wait before trying again.
    at Supabase_VectorStores.upsert (/usr/local/lib/node_modules/flowise/node_modules/flowise-components/dist/nodes/vectorstores/Supabase/Supabase.js:37:27)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async buildFlow (/usr/local/lib/node_modules/flowise/dist/utils/index.js:265:17)
    at async App.upsertVector (/usr/local/lib/node_modules/flowise/dist/index.js:1698:13)
    at async /usr/local/lib/node_modules/flowise/dist/index.js:1193:13
2024-04-01 20:38:00 [ERROR]: [server]: Error: Error: ThrottlingException: Too many requests, please wait before trying again. You have sent too many requests.  Wait before trying again.
Error: Error: ThrottlingException: Too many requests, please wait before trying again. You have sent too many requests.  Wait before trying again.
    at buildFlow (/usr/local/lib/node_modules/flowise/dist/utils/index.js:326:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async App.upsertVector (/usr/local/lib/node_modules/flowise/dist/index.js:1698:13)
    at async /usr/local/lib/node_modules/flowise/dist/index.js:1193:13

To Reproduce Steps to reproduce the behavior:

1. Select JSON file (mine is 4MB)
2. Click Upsert Button

Expected behavior The Upsert to complete successfully, after the objects are enqueued and processed.

Screenshots

Setup

• docker
• Flowise Version 1.6.2
• OS: Linux
• Browser Brave

[frankyw]

Also tried with a text file of the same data (3.5MB), with the same result, so looks like some type of Amazon-specific socket usage bug? Perhaps 50 sockets is too high?

2024-04-01 21:38:18 [INFO]: ⬆️ POST /api/v1/vector/internal-upsert/8bccc61b-3401-445b-9b8f-431f2c34a15b
@smithy/node-http-handler:WARN socket usage at capacity=50 and 4091 additional requests are enqueued. See https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/node-configuring-maxsockets.html or increase socketAcquisitionWarningTimeout=(millis) in the NodeHttpHandler config.
2024-04-01 21:38:31 [ERROR]: ThrottlingException: Too many requests, please wait before trying again. You have sent too many requests.  Wait before trying again.
Error: ThrottlingException: Too many requests, please wait before trying again. You have sent too many requests.  Wait before trying again.
    at Supabase_VectorStores.upsert (/usr/local/lib/node_modules/flowise/node_modules/flowise-components/dist/nodes/vectorstores/Supabase/Supabase.js:37:27)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async buildFlow (/usr/local/lib/node_modules/flowise/dist/utils/index.js:265:17)
    at async App.upsertVector (/usr/local/lib/node_modules/flowise/dist/index.js:1698:13)
    at async /usr/local/lib/node_modules/flowise/dist/index.js:1193:13
2024-04-01 21:38:31 [ERROR]: [server]: Error: Error: ThrottlingException: Too many requests, please wait before trying again. You have sent too many requests.  Wait before trying again.
Error: Error: ThrottlingException: Too many requests, please wait before trying again. You have sent too many requests.  Wait before trying again.
    at buildFlow (/usr/local/lib/node_modules/flowise/dist/utils/index.js:326:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async App.upsertVector (/usr/local/lib/node_modules/flowise/dist/index.js:1698:13)
    at async /usr/local/lib/node_modules/flowise/dist/index.js:1193:13

[HenryHengZJ]

I tried using amazon.titan-embed-g1-text-02 with a 12mb PDF file, it was working fine:

I'd suggest try changing vector store, embedding model to see which one was causing the error

[haimco50]

I started having the same issue as well when I upgraded my Bedrock emnedding model and my Bedrock LLM model to titan-text-v2 embedding and Cohere 3 - after spending some time on it I think I was able to pin point the issue to the embedding model - basically you are going to have this issue for any Bedrock model that isn't amazon.titan-embed-text-v1 (I Tried on Cohere and Titan-text V2) - once I downgraded the embedding model to V1 the issue was gone.

[HenryHengZJ]

should be fixed via PR with the support of cohere embed in bedrock