Multithreading, how it can handle many async requests at the same time?

[GabrielBrotas]

Details

Hi Guys, I understand that Node.js uses a single-thread and an event loop to process requests only processing one at a time (which is non-blocking) and all async functions (I/O operations, File System, DNS resolution,...) goes to another internal thread pool (which has 4 threads by default, right?) to be resolved and then return the response in the callback to the event loop. But still, how does that work, lets say 10,000 concurrent requests to an api route that has to wait a response from another api. since it has just 1 thread to dns lookup, does that mean this thread will be blocked? The event loop will process all the requests? the thread pool will have to wait request by request to return the response via callback? Is there a way to create an elastic thread poll? so we can have 1 thread for each async operation?

Node.js version

Not applicable.

Example code

No response

Operating system

Linux

Scope

Scale

Module and version

Not applicable.

[devsnek]

At a high level, the thread pool is only used to make blocking operations appear asynchronous (for example, non-io_uring fs operations on linux are synchronous). tcp/udp can be used in a real non-blocking mode, so adding more threads won't help at all with that.

[GabrielBrotas]

If I add an api request to my app, eg:

  app.get('/', async (req, res) => {
  ...
  const result = await axios.get('www.someapi.com')
  ...
  res.status(200).json({ result })
  })

this axios.get will be executed in the thread pool, right? because has the dns resolution

So the question is, let's suppose this request takes 2 seconds to resolve, if nodejs only has 4 threads in the internal thread pool, how can my route / handle 1000 concurrent requests without blocking?

[ErickWendel]

I did a very interesting video about this subject if you wanna check it out

The magic behind Node.js is that Node.js schedules a few threads to do all processing. Scheduling more threads means more memory usage and it's not actually something that will help you to process faster operations.

In your example, imagine that for each axis request you have it takes 5 seconds to respond.

You said you would have 10K requests per second

There, Node.js will schedule async tasks which will distribute across those 4 threads.

Node.js itself won't be blocked by those async tasks. it'll have a bunch of pending tasks to resolve and when they respond Node.js will receive the callback and keep that specific request working.

What blocks node.js is not the amount of async tasks you do but synchronous code. Such as While Loops, imperative code, parsing, Regex and much more.

If you wanna experiment your self you can:

1. install autocannon - a Node.js package for load testing
2. install clinicjs - a Node.js package for analysing the Node.js health

Here a tutorial for it from the official docs: https://clinicjs.org/documentation/flame/03-first-analysis/

Please, if it answered your question close the issue

[GabenGar]

Are While Loops still blocking when ran within async functions?

[Amit006]

@GabenGar
Yes, while loops can still block when run within async functions if they are not properly managed.

In JavaScript, async functions allow you to write asynchronous code using await expressions, which can pause the execution of the function until a promise is resolved. However, if you use a while loop inside an async function without yielding control back to the event loop, it can block the execution of other asynchronous tasks.

Here's an example to illustrate this:

javascript Copy code async function asyncFunction() { let i = 0; while (i < 10) { console.log(i); i++; } console.log('Finished loop'); }

asyncFunction(); console.log('Async function called'); In this example, the asyncFunction contains a while loop that iterates from 0 to 9. When you call asyncFunction, it starts executing, and the while loop blocks the execution until it finishes iterating through all the numbers. As a result, the output will be: 0 1 2 3 4 5 6 7 8 9 Finished loop Async function called The console.log('Async function called') statement is only executed after the while loop finishes, even though asyncFunction is declared as an async function.

To avoid blocking the event loop, you should use asynchronous constructs like await within the while loop to yield control back to the event loop. For example, you could use await with a setTimeout to simulate asynchronous behavior:

async function asyncFunction() { let i = 0; while (i < 10) { console.log(i); await new Promise(resolve => setTimeout(resolve, 1000)); // Simulate asynchronous behavior i++; } console.log('Finished loop'); }

asyncFunction(); console.log('Async function called'); In this modified example, await new Promise(resolve => setTimeout(resolve, 1000)) pauses the execution of the loop for 1 second in each iteration, allowing other asynchronous tasks to run concurrently. As a result, the output will be more asynchronous, with a 1-second delay between each number printed in the loop.