node-solid-server icon indicating copy to clipboard operation
node-solid-server copied to clipboard

Server limitations when querying multiple pods (>128) in parallel

Open Ga11u opened this issue 4 years ago • 2 comments

Hi,

I am running some experiments querying more than 128 pods in parallel from the Inrupt server and SolidCommunity. In both cases when I query data from more than 128 pods (i.e., 256, 512, 1024) at the same time the server stops the queries and blocks them.

When I experiment with 64 or fewer pods there is no problem. However, I have to do it in batches of 64 pods when I want to query more than 64 pods in parallel otherwise the server starts blocking my HTTP requests.

Is it expected that a SOLID server block parallel queries for a large number of pods from the same client?

Thank you

Crossposted: https://github.com/inrupt/pod-server/issues/76

Ga11u avatar May 10 '21 10:05 Ga11u

SolidCommunity

Note that https://solidcommunity.net/ runs Node Solid Server.

RubenVerborgh avatar May 10 '21 10:05 RubenVerborgh

@RubenVerborgh I was checking if this could be something related with NodeJS server, but Node is suposed to be able to handle more that 1000 cocurrent requests ... I am still trying to understand why it blocks the calls

Ga11u avatar May 26 '21 12:05 Ga11u