Reload taking long time & blocking other web requests

[eleksis]

Hi, having some performance issues with oxidized. If someone could help, what could be the problem or how we can mitigate this.

We have reload interval set to 60s, to update/refresh device list from a file. Total of around +4000 devices.

Looking at the code...reload is triggered via curl -s http://localhost:8888/reload?format=json

This operation in our case takes 60-90s. Meanwhile all other requests to web are blocked. We are calling /node/next/devicex.json from other aplication to force update some devices and these requests timeout.

During reload it looks like oxidized is doing dns resolvements, as there are alwasy a few ResolvError with message 'no address <fqdn>'

Questions that come to my mind:

Why reload takes so long? Might dns resolvements be a problem? And so, we can disable this It there only one thread to serve requests? How can we adjust setup/config to mitigate this?

Thanks for any advices.

[laf]

If you are using http/sql as a backend then this pr added the ability to reload just a single device (both new and existing). Might be better to do it that way everytime you add or update a device?

https://github.com/ytti/oxidized-web/pull/206