fox-it
Duplicate keys in Netplan configuration results in errors
The target is a Velociraptor collection of ARM Ubuntu 24.04.2 LTS (Noble Numbat).
Executing the following command results in an error because the Netplan configuration contains duplicate keys.
target-query -t <TARGET> -f ips
XXXX [error ] <Target <TARGET>: Exception while executing function ips (os.unix.linux.debian._os.ips): while constructing a mapping
in "<file>", line 2, column 3
found duplicate key "ethernets" with value "{}" (original value: "{}")
in "<file>", line 6, column 3
To suppress this check see:
https://yaml.dev/doc/ruamel.yaml/api/#Duplicate_keys
The Netplan configuration with the duplicate key ethernets. The configuration was verified with netplan status and netplan apply.
network:
version: 2
ethernets:
ens33:
dhcp4: true
ethernets:
ens34:
dhcp4: no
addresses: [172.16.0.12/24]
The solution provided by ruamel is not sufficient because its discards the second key.
It is an error for two equal keys to appear in the same mapping node. In such a case the YAML processor may continue, ignoring the second key: value pair and issuing an appropriate warning. This strategy preserves a consistent information model for one-pass and random access applications. - https://yaml.org/spec/1.1/#id932806
Ruamel is not sufficient enough and I guess writing our own yaml parser is not intended. Nonetheless, Ruamel can't be blamed because the valid Netplan configuration does not adhere the YAML specification.
@Schamper do you have a suggestion on how to fix this issue before I start working on it?
I'd rather not switch back to PyYAML. I see in the README of ruamel.yaml that they'll also remove the allowing of duplicate keys completely in the next release...
Is there some kind of monkeypatching we can do in ruamel.yaml? Or maybe inspecting the netplan config beforehand and patching YAML merge keys in duplicate keys (determined by a simple line-by-line check function)?
@respondersGY if you do work on this, can you please not fix it in the network_managers.py implementation, but rather convert it to the new NetworkPlugin interface?
I currently don't have time to work on this in the upcoming weeks, but I quickly looked into it.
I don't understand where the fix should be implemented in the NetworkPlguin interface in comparison to network_managers.py.
The LinuxPlugin has the function dhcp which calls LinuxNetworkManager, but the NetworkPlugin also has a function named dhcp which calls LinuxNetworkConfigParser. What is the difference between network_managers.py and network.py? The current setup feels more complex than it should be.
network_manager.py and any and all references or calls into it will be completely removed in a future release. Everything will have to be implemented as a NetworkPlugin.