REST API Inventory Better Handle Large Inventories

[jvanderaa]

ISSUE TYPE

• Bug Report

SOFTWARE VERSIONS

pynautobot

N/A - REST API Inventory

Ansible:

x.x

Nautobot:

1.1.4

Collection:

networktocode.nautobot

SUMMARY

With large inventories, Ansible may cause requests to timeout for Nautobot due to asking for all resources.

STEPS TO REPRODUCE

1. Create a large inventory in Nautobot
2. Use Nautobot Inventory
3. See timeouts and large CPU utilization

EXPECTED RESULTS

Completed inventory

ACTUAL RESULTS

Incomplete inventory

Possible Solution

1. Look into handling pagination built into Nautobot instead of requesting all
2. Evaluate moving to pynautobot to handle pagination
3. Others

[nniehoff]

Another possible solution would be to investigate the use of GraphQL here.

[jvanderaa]

GraphQL inventory is the absolutely recommended methodology @nniehoff. We recommend keeping an inventory light and with only what is required to connect to the device. This way it is efficient and nimble. For example, not all playbook executions will need interface data. So why even have that in inventories. The playbook that needs to have the interface data should gather the data as needed from a SOT when required.

[itdependsnetworks]

There is pagination for device, but not the others, such as interfaces. While in general we should probably warn against using interfaces and many other data points, that being said, we should optimize in the same way in all scenarios.

[jvanderaa]

I'm going to get this covered with documentation. I'd rather not put a warning in that then people may try to supress.