FlowiseAI
[BUG] Error: Error: 400 Bad Request: Payload error: JSON payload (37327342 bytes) is larger than allowed (limit: 33554432 bytes).
Im trying to embed a vector db (quadrant) with langchain docs with sitemap.xml link extraction.
everything works fine and the embedding starts, but after a while I get this error:
Error: Error: 400 Bad Request: Payload error: JSON payload (37327342 bytes) is larger than allowed (limit: 33554432 bytes).
is it possible to increase the JSON payload so it can manage all links?
Is the error coming from Qdrant not able to handle too large of a payload? Or its from Flowise side that is not able to pass the payload over to Qdrant?
I got the same error, I think the qdrant client must send the data in batch mode, and I think we must be able to customize the batch_size in the qdrant node
@vipervs: solution found for the problem. plz check https://github.com/FlowiseAI/Flowise/issues/2191
Increase QDrant limit size is not a solution. I am trying to upload 211kb csv file and I am getting the same error. Very strange behaviour.
@xmaiconx: increase the max file upload size for Qdrant in the custom config.yaml for Qdrant as per ur use case...this is not a limitation from flowise but Qdrant...i faced problem with a txt file of less than 1 mb. use the config.yaml for configuring Qdrant to accept big files. size of the source file is not the issue but the no of vectors that are being generated from the content of the file that is being pushed in Qdrant in a single go is the bottleneck...Qdrant by default accepts max 32 MB (33554432 bytes) in a single go...increase that size as per ur requirement and the problem goes away.