Support bulk operations

[lampwins]

Environment

• Nautobot version: 1.2.5
• nautobot-ssot version: 1.1.0

Proposed Functionality

As a Tools Integrator, I want the SSoT plugin to support bulk operations within the nautobot ORM (or other target systems which support like functionality) so that jobs that deal with a lot of data can operate more performantly.

Use Case

I have several use cases which involve full inventory syncs between systems involving upwards of 75k+ objects. The run time of these jobs is currently measured in hours due to the serial nature of processing objects after the diff is calculated.

There is a tradeoff in nautobot concerning bulk ORM operations (bulk_create, bulk_update, etc) in terms of change logging and other signally, but in my case, I am willing to accept those downsides to be able to process the SSoT sync with better performance.

[glennmatthews]

To some extent it may be possible to implement this by implementing deferred create/update/delete operations and using the DiffSync sync_complete callback to perform the requisite bulk operations. See https://blog.networktocode.com/post/advanced-ssot-app/ for some simple examples.

[chadell]

@lampwins @glennmatthews are there any plans in Nautobot Core to support ORM bulk operations while updating the changelog?

[glennmatthews]

Non-trivial to support since by design ORM bulk operations do not trigger signals.