napalm icon indicating copy to clipboard operation
napalm copied to clipboard

Napalm validate inside list search for patterns instead of fullmatch

Open Tristou27 opened this issue 11 months ago • 2 comments

Description of Issue/Question

When using the validate / compliance_report function, the list validation is not done properly. For a given "item" in a validation file, if this item is specified as a list, then the _compare_getter_list() method is called. This method used compare() for each element in the list and if the elements are strings, then the re.search() method is used.

The issue is by doing that, false positive can be found as the re.search() method only look for a partial match. For example, if we have a validation file checking for "Vlan5" interface in the interface_list item, and that the device sends "Vlan5554", it will be considered as a positive result even though it's not.

A quick "fix" I can think of is the use of re.fullmatch() instead of re.search(). Or introduce a specific syntax in the validation file to precise if we're talking about pattern or "complete strings".

Did you follow the steps from https://github.com/napalm-automation/napalm#faq

(Place an x between the square brackets where applicable)

  • [x] Yes
  • [] No

Setup

napalm version

(Paste verbatim output from pip freeze | grep napalm between quotes below)

# ttr @ serverin ~/autotest on git:main x .venv39 [17:42:42] 
$ pip freeze | grep napalm
napalm==4.0.0

Network operating system version

(Paste verbatim output from show version - or equivalent - between quotes below)

N/A

Steps to Reproduce the Issue

  1. Build a validation file for get_facts, check if the "interface_list" attribute contains "Vlan5" entry.
  2. Configure a device where a interface called Vlan5553 is created
  3. Run the validation, it will be detected as positive even though Vlan5 interface is not created on the device.

Tristou27 avatar Mar 20 '24 16:03 Tristou27