network-importer icon indicating copy to clipboard operation
network-importer copied to clipboard

invalid DiffElement - just fresh install of network-importer-3.0.3

Open hikarikumo opened this issue 3 years ago • 5 comments
trafficstars

Environment

  • Python version: Python 3.8.10
  • Network Importer version: 3.0.3

Configuration

[main] import_ips = true import_prefixes = true import_intf_status = false import_vlans = "config" import_cabling = "config" excluded_platforms_cabling = ["cisco_asa"] nbr_workers = 25 configs_directory = "configs" backend = "nautobot" [inventory] supported_platforms = [ "cisco_ios", "cisco_nxos", "arista_eos" ] [inventory.settings] address = "" token = "" verify_ssl = false [batfish] address= "localhost" network_name = "network-importer" # Alternative Env Variable : BATFISH_NETWORK_NAME port_v1 = 9997 # Alternative Env Variable : BATFISH_PORT_V1 port_v2 = 9996 # Alternative Env Variable : BATFISH_PORT_V2 use_ssl = false # Alternative Env Variable : BATFISH_USE_SSL [network] login = "cisco" # Alternative Env Variable : NETWORK_DEVICE_LOGIN password = "cisco" # Alternative Env Variable : NETWORK_DEVICE_PWD enable = true # Alternative Env Variable : NETWORK_DEVICE_ENABLE fqdns = [ ] [network.netmiko_extras] [network.napalm_extras] [drivers.mapping] default = "network_importer.drivers.default" cisco_nxos = "network_importer.drivers.cisco_default" cisco_ios = "network_importer.drivers.cisco_default" cisco_xr = "network_importer.drivers.cisco_default" juniper_junos = "network_importer.drivers.juniper_junos" arista_eos = "network_importer.drivers.arista_eos" [logs] level = "info" # "debug", "info", "warning" performance_log = false performance_log_directory = "performance_logs"

Steps to Reproduce

  1. install network-importer
  2. created configuration
  3. run network-importer check --update-configs

Expected Behavior

It was expected to receive valid output, however, I had received the errors instead

Observed Behavior

Traceback (network-importer-venv) ✔ ~/git/network-importer-test [main|✔] 15:02 $ network-importer check 2022-02-07 15:02:45,333 - network-importer - INFO - Import SOT Model 2022-02-07 15:02:47,255 - network-importer - INFO - Import Network Model Traceback (most recent call last): File "/home/hikari/network-importer-venv/bin/network-importer", line 8, in sys.exit(main()) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 829, in call return self.main(*args, **kwargs) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/network_importer/cli.py", line 149, in check diff = ni.diff() File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/network_importer/main.py", line 146, in diff return self.sot.diff_from(self.network, diff_class=NetworkImporterDiff) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/init.py", line 594, in diff_from return differ.calculate_diffs() File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/helpers.py", line 87, in calculate_diffs diff_elements = self.diff_object_list( File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/helpers.py", line 132, in diff_object_list if diff_element: File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/diff.py", line 219, in len for child in self.get_children(): File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/diff.py", line 306, in get_children yield from self.child_diff.get_children() File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/diff.py", line 95, in get_children yield from order_method(self.children[group]) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/network_importer/diff.py", line 53, in order_children_interface raise Exception("invalid DiffElement") Exception: invalid DiffElement

hikarikumo avatar Feb 07 '22 15:02 hikarikumo

I 'll second that, I am facing the exact same situation.

Same python version, same network importer version.

[edit]

A quick and dirty fix that seems to work for me is the following

            action = child.action

            if not action:
                action = "update"
            else:
                action = child.action.value

nstamoul avatar Feb 13 '22 09:02 nstamoul

hello @nstamoul Could you please guide me on what exactly has to be changed? I have changed it into the following ' def order_children_interface(cls, children): """Return the interface children ordered order.""" intfs_lags = defaultdict(list) intfs_regs = defaultdict(list) intfs_lag_members = defaultdict(list)

    for child_name, child in children.items():
        action = child.action
        if not action:
            action = "update"
        if action == "delete":
            if "is_lag" in child.dest_attrs and child.dest_attrs["is_lag"]:
                intfs_lags[action].append(child_name)
            elif "is_lag_member" in child.dest_attrs and child.dest_attrs["is_lag_member"]:
                intfs_lag_members[action].append(child_name)
            else:
                intfs_regs[action].append(child_name)
        elif action in ["update", "create"]:
            if "is_lag" in child.source_attrs and child.source_attrs["is_lag"]:
                intfs_lags[action].append(child_name)
            elif "is_lag_member" in child.source_attrs and child.source_attrs["is_lag_member"]:
                intfs_lag_members[action].append(child_name)
            else:
                intfs_regs[action].append(child_name)
        else:
            action = child.action.value
            # raise Exception("invalid DiffElement")

' The check now runs fine, however apply fails: Check $ network-importer check --update-configs 2022-02-15 08:33:22,070 - network-importer - INFO - Updating configuration from devices .. 2022-02-15 08:33:23,055 - network-importer - INFO - R1 | Latest config file already present ... 2022-02-15 08:33:23,132 - network-importer - INFO - R3 | Latest config file already present ... 2022-02-15 08:33:23,133 - network-importer - INFO - Import SOT Model 2022-02-15 08:33:23,971 - network-importer - INFO - Import Network Model site site: home vlan vlan: home__5 MISSING in Network vlan: home__6 MISSING in Network prefix prefix: home__172.16.244.0/24 MISSING in Nautobot prefix: home__192.168.100.0/24 MISSING in Nautobot device device: R1 interface interface: Ethernet0/1 MISSING in Nautobot interface: Ethernet0/2 MISSING in Nautobot interface: Ethernet0/0 ip_address ip_address: R1__Ethernet0/0__172.16.244.101/24 MISSING in Nautobot interface: Ethernet0/3 MISSING in Nautobot device: R3 interface interface: Ethernet0/3 MISSING in Nautobot interface: Ethernet0/1 MISSING in Nautobot interface: Ethernet0/2 MISSING in Nautobot 2022-02-15 08:33:26,586 - network-importer - INFO - Execution finished, processed 2 device(s)

Apply $ network-importer apply 2022-02-15 08:33:56,032 - network-importer - INFO - Import SOT Model 2022-02-15 08:33:56,710 - network-importer - INFO - Import Network Model Traceback (most recent call last): File "/home/hikari/network-importer-venv/bin/network-importer", line 8, in sys.exit(main()) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 829, in call return self.main(*args, **kwargs) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/network_importer/cli.py", line 108, in apply ni.sync() File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/network_importer/main.py", line 142, in sync self.sot.sync_from(self.network, diff_class=NetworkImporterDiff) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/init.py", line 525, in sync_from result = syncer.perform_sync() File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/helpers.py", line 326, in perform_sync changed |= self.sync_diff_element(element) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/helpers.py", line 390, in sync_diff_element changed |= self.sync_diff_element(child, parent_model=dst_model) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/helpers.py", line 366, in sync_diff_element changed, modified_model = self.sync_model(src_model=src_model, dst_model=dst_model, ids=ids, attrs=attrs) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/diffsync/helpers.py", line 415, in sync_model dst_model = self.model_class.create(diffsync=self.dst_diffsync, ids=ids, attrs=attrs) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/network_importer/adapters/nautobot_api/models.py", line 436, in create LOGGER.info("Created Prefix %s (%s) in Nautobot", prefix.prefix, prefix.id) File "/home/hikari/network-importer-venv/lib/python3.8/site-packages/pynautobot/core/response.py", line 192, in getattr raise AttributeError('object has no attribute "{}"'.format(k)) AttributeError: object has no attribute "prefix"

hikarikumo avatar Feb 15 '22 08:02 hikarikumo

Hi @hikarikumo , that seems to be a different issue, so I am afraid I can't really help you there.

You could always try troubleshooting the problem, starting by finding out what kind of object k is.

nstamoul avatar Feb 15 '22 21:02 nstamoul

Hi @hikarikumo , that seems to be a different issue, so I am afraid I can't really help you there.

You could always try troubleshooting the problem, starting by finding out what kind of object k is.

nstamoul avatar Feb 15 '22 21:02 nstamoul

This appears to be due to a breaking change in diffsync 1.4.x (https://github.com/networktocode/diffsync/pull/90). If that's the case, doing a pip install diffsync==1.3.0 should resolve this issue until this library can be updated to account for that change.

glennmatthews avatar Mar 01 '22 15:03 glennmatthews