[Bug]: Inventory Module crash if fetch_all is False and prefixes is True

[BeArchiTek]

Ansible NetBox Collection version

3.5.1

Ansible version

ansible [core 2.12.2]
  config file = /root/ansible/ansible.cfg
  configured module search path = ['/root/ansible/library']
  ansible python module location = /usr/local/lib/python3.9/dist-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.2 (default, Feb 28 2021, 17:03:44) [GCC 10.2.1 20210110]
  jinja version = 3.0.3
  libyaml = True

NetBox version

3.1.5

Python version

3.9

Steps to Reproduce

1. Create a new inventory file using netbox..netbox.nb_inventory plugin

plugin: netbox.netbox.nb_inventory
api_endpoint: https://netbox-url
validate_certs: True

# Limit what we fetch
services: False
interfaces: False
fetch_all: False

# Fetch extra Data structure
site_data: True
prefixes: True

group_by:
  - role
  - site

compose:
  # Composition using device informations
  ansible_network_os: platform.slug

query_filters:
  - role: foo
  - role: bar

2. Run the inventory to get the devices from netbox

ansible-inventory -i netbox_inventory.yml -y --list --output inventories/production.yml

Expected Behavior

Be able to fetch the prefixes

Observed Behavior

An Exception is raise :

Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/root/.ansible/collections/ansible_collections/netbox/netbox/plugins/inventory/nb_inventory.py", line 1274, in wrapper
    raise e
  File "/root/.ansible/collections/ansible_collections/netbox/netbox/plugins/inventory/nb_inventory.py", line 1269, in wrapper
    lookup()
  File "/root/.ansible/collections/ansible_collections/netbox/netbox/plugins/inventory/nb_inventory.py", line 904, in refresh_prefixes
    query_values=list(self.sites_with_prefixes),
AttributeError: 'InventoryModule' object has no attribute 'sites_with_prefixes'

[BeArchiTek]

Same behaviour with netbox 3.2.6 with collection 3.7.1

Ansible NetBox Collection version

3.7.1

Ansible version

ansible [core 2.13.1]
  config file = /root/ansible/ansible.cfg
  configured module search path = ['/root/ansible/library']
  ansible python module location = /usr/local/lib/python3.9/dist-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.9.2 (default, Feb 28 2021, 17:03:44) [GCC 10.2.1 20210110]
  jinja version = 3.1.2
  libyaml = True

NetBox version

3.2.6

Python version

3.9