netbox-community
no cluster list with a large number of virtual servers
Deployment Type
Self-hosted
NetBox Version
v4.2.2
Python Version
3.11
Steps to Reproduce
add a large number of virtual servers (for ex 100К)
- select .../virtualization/virtual-machines/
- select Filters
- click inside Cluster box.
- see "No results found"
Expected Behavior
list of clusters
Observed Behavior
no list of clusters
Thank you for opening a bug report, @someconservator . Unfortunately, the information you have provided is not sufficient for someone else to attempt to reproduce the reported behavior. Each bug report must include detailed steps that someone else can follow on a clean, empty NetBox installation to reproduce the exact problem you're experiencing.
These instructions should include the creation of any involved objects, any configuration changes, and complete accounting of the actions being taken. Be sure that your report does not reference data on the public NetBox demo, as that is subject to change at any time by an outside party and cannot be relied upon for bug reports.
Additionally, I note that this is reported against v4.2.2. Please verify the behavior in the current release (v4.3.2).
O! Sorry! Additional steps: a) the respective clusters mast be created. b) most of those 100K+ servers must be associated with one of them.
Also, we can't repeat this error on the latest working version yet due to the large number of netbox users who take a long time to approve upgrades. and i'm afraid we'll always be behind. i reported this just in case, suddenly nothing has changed in this place in the new versions. Just reported what I could. Maybe it will be useful...
By the way, reducing the number of servers predictably removed the problem, so I believe it is somehow related to the ORM and and presumably with the fact that netbox makes here its request not just for clusters, but also for all servers associated with them.
@someconservator could you please use your browser's development tools or some other tooling to inspect the API request and confirm whether it is returning a valid response?
Request URL https://gs-ltisntbox02.isb/api/virtualization/clusters/?brief=true&limit=100 Request Method GET Status Code 504 Gateway Time-out
@someconservator , this sounds like an issue with the local configuration of your NetBox instance, as now gateway is in use in the core project and should never return a 504.
I'm going to close this as wontfix, but you might want to post in the discussion forum with more details about your setup and see if someone in the community can assist.
as soon as I reduce the number of records in the database, the error disappears. could you suggest how this might be related to local netbox configuration?
You likely need to extend the maximum request time configured for your WSGI server. For further assistance, please start a discussion.
Yes, I agree, it can de done, but don't you think, Jeremy, that the tying the list of virtualization clusters to the number of instances they contain is a bit excessive in the context of filter construction? The user wants to limit the list of instances by selecting a cluster, but instead implicitly gets the whole insance list and forced to wait a long time, while at filter construction time, all one needs is a simple list from the cluster table (well, maybe more precisely, the cluster class in Django terms).
Again, let's assume we have 200K (or even 500) servers instead of 100K. How much should we increase the timeout, add RAM to the server, and ask the user to wait for a response each time? In theory, choosing a cluster when constructing a filter for VMs should be a short time operation...