snmp_notifier icon indicating copy to clipboard operation
snmp_notifier copied to clipboard

Object order not respected

Open fabienmagagnosc opened this issue 1 year ago • 6 comments

What did you do?

I added 4 templates to the original trap, to provide more data to the SNMP system

What did you expect to see?

a simple trap with the trap object ID respected

What did you see instead? Under which circumstances?

the trap is generated, but the order of the ObjectID are not respected, causing parsing issue by the libreNM and openNMS

Environment

docker/latest SNMP notifier

  • System information:

    not applicable

  • SNMP notifier version:

~ $ snmp_notifier --version --help snmp_notifier, version 1.5.0 (branch: HEAD, revision: af4f2a9f10e351b98694969e232771f7f981af81) build user: root@b0fff167f945 build date: 20231105-08:03:08 go version: go1.21.3 platform: linux/amd64 tags: netgo

  • Alertmanager version:

/alertmanager $ alertmanager --version alertmanager, version 0.27.0 (branch: HEAD, revision: 0aa3c2aad14cff039931923ab16b26b7481783b5) build user: root@22cd11f671e9 build date: 20240228-11:51:20 go version: go1.21.7 platform: linux/amd64 tags: netgo

  • Prometheus version:

    N/A

  • Alertmanager command line:

    N/A
    
  • SNMP notifier command line:

    volumes: - /opt/tecnotree/containers/persistence/insights-ccbu-alertmanager-snmp/etc/snmp_notifier:/etc/snmp_notifier ports: - "9464:9464" command: - "--snmp.trap-description-template=/etc/snmp_notifier/description-template.tpl" - "--snmp.extra-field-template=4=/etc/snmp_notifier/instance-template.tpl" - "--snmp.extra-field-template=5=/etc/snmp_notifier/summary-template.tpl" - "--snmp.extra-field-template=6=/etc/snmp_notifier/status-template.tpl" - "--snmp.extra-field-template=7=/etc/snmp_notifier/job-template.tpl" - "--snmp.extra-field-template=8=/etc/snmp_notifier/comment-template.tpl" - "--snmp.version=V2c" - "--snmp.destination=127.0.0.1:162" - "--snmp.community=system" - "--snmp.timeout=5s" - "--alert.severity-label=severity" - "--alert.default-severity=critical" - "--alert.severities=critical,warning,info"

  • Prometheus alert file:

    N/A

  • Logs:

image

fabienmagagnosc avatar Nov 20 '24 10:11 fabienmagagnosc

and funnily, when I set the order to follow the failed order .. it follows the order.

  - "--snmp.trap-description-template=/etc/snmp_notifier/description-template.tpl"
  - "--snmp.extra-field-template=4=/etc/snmp_notifier/summary-template.tpl"
  - "--snmp.extra-field-template=5=/etc/snmp_notifier/status-template.tpl"
  - "--snmp.extra-field-template=6=/etc/snmp_notifier/job-template.tpl"
  - "--snmp.extra-field-template=7=/etc/snmp_notifier/comment-template.tpl"
  - "--snmp.extra-field-template=8=/etc/snmp_notifier/instance-template.tpl"

image

I suspect hat for performance, the notfied spawn all the calculation in parallel, and then merge them. But the order is not respected, but "added" one after another one causing the ordering failure.

fabienmagagnosc avatar Nov 20 '24 10:11 fabienmagagnosc

but it seems anyway, that the execution is "too random" and cause a lot of problem as this is not stabel for any NMS to process.

in my case, I managed to get it parsed properly, one every 5 traps

fabienmagagnosc avatar Nov 20 '24 10:11 fabienmagagnosc

it seems that the addition of the fields inside the traps is not "order incrementally"

for subOid, template := range trapSender.configuration.ExtraFieldTemplates {

fabienmagagnosc avatar Nov 20 '24 10:11 fabienmagagnosc

those extra fields are important, as they allow to carry way more information, in my case :

  • summary of the incident (from the Alarm Annotations Summary)
  • a status OK/FAULT to start/stop the alarm
  • the job to identify the type of alarms generated and linked directly to the Prometheus group
  • an optional comment to add resolution/command line
  • the instance to be sure it's identified inside the NMS automatically and change the instance/server color according to the criticality

fabienmagagnosc avatar Nov 20 '24 10:11 fabienmagagnosc

it seems that the addition of the fields inside the traps is not "order incrementally"

for subOid, template := range trapSender.configuration.ExtraFieldTemplates {

True. I will ensure it is sorted by the OID.

maxwo avatar Jan 26 '25 20:01 maxwo

i did an internal version for testing. sorry if I didn't reverted back and helped

in the trap_sender.go. I imported :

"sort"

and changed the way the OID are added to the trap.

_// Extract the keys from the map
var subOids []string
for subOid := range trapSender.configuration.ExtraFieldTemplates {
	subOids = append(subOids, subOid)
}

// Sort the keys
sort.Strings(subOids) 

// Iterate over the sorted keys
for _, subOid := range subOids {
	template := trapSender.configuration.ExtraFieldTemplates[subOid]_

attached my result

trap_sender.go.txt

fabienmagagnosc avatar Jan 29 '25 17:01 fabienmagagnosc

Issue is solved: it was due to the use of an hash-map for reading command line arguments, which does not guarantee the order of elements.

I will release after another new feature is done.

maxwo avatar Apr 15 '25 05:04 maxwo

@fabienmagagnosc FYI, fix has been released in 1.6.2

maxwo avatar Apr 21 '25 12:04 maxwo