Nomad 1.8.3 - Unable to navigate from Clients to Jobs in the UI with more than 5 clients

[ngcmac]

Nomad version

1.8.3

Operating system and Environment details

EC2 Debian 12

Issue

After we upgrade our pre-production cluster (3 servers + 12 clients) from 1.8.2 to 1.8.3, we are unable to navigate in the UI from Clients to Storage or Clients to Jobs.

When we navigate to Clients, there's a request that appears to be trying to fetch all the allocation per client that's stuck in pending state:

I don't know if this is the expected behavior or also a bug, but looking to some other clusters running version 1.6.3, we observe the same thing, a request per node in pending state after we navigate to Clients. The difference is that when we then try to navigate away from Clients to Jobs in version prior to 1.8.3, the pending requests are canceled and we are able to display the Jobs page:

In version 1.8.3, the pending requests are not cancelled after we click in Jobs and the new request is also in pending status:

The browser keeps unresponsive, the route/path in the URL doesn't change. Reload doesn't work. We need to navigate to some other page (Clients to Servers for example) and then navigate to Jobs again for it to works. If we change to a different tab in the browser while unresponsive and then came back, the Jobs page will be displayed.

This only happens when we try to navigate from Clients to Jobs ou from Clients to Storage. Any other combination seams to be working, all the pending request are canceled and the page is displayed. Also, this only happens if the number of clients is greater than 5 (state doesn't matter). If we filter the nodes to show less than 5, the navigation from Clients to Jobs works as expected. Can it be related with max number of TCP connections to the same host in the browser (6 in FF/Chrome)??

At first we though that it could be proxy related (using apache2 as reverse proxy to Nomad) but accessing directly to any of the servers, we have the same behavior.

Reproduction steps

Access the Nomad's UI in a server that has more than 5 Nomad Clients. Nomad version should be 1.8.3. Navigate to Clients. Then try to navigate to Jobs. It doesn't work.

Expected Result

Access the Nomad's UI in a server that has more than 5 Nomad Clients. Nomad version should be 1.8.3. Navigate to Clients. Then try to navigate to Jobs. It works and Jobs page is displayed .

Actual Result

Job file (if appropriate)

Nomad Server logs (if appropriate)

Nothing relevant in the logs

Nomad Client logs (if appropriate)

Nothing relevant in the logs