feat: Add embedded_tags functionality to gNMI plugin

[cprecup]

The approach is needed because YANG operational models can contain key-less elements, i.e., elements in a list that have no unique identifier. The approach was tested on Cisco IOS-XR.

Pull request addresses the following two issues:

• Resolves #8250
• Resolves #7462
• Resolves #7805

[cprecup]

@ssoroka Thank you for your review. I've just committed a change with a new test dedicated to this feature.

[bewing]

Hi! Thanks for taking the time to work on this issue!

I'm having some issue getting configuration to work to conform to my use case: Adding interface descriptions as tags to interface counters. I've tried the following two configs:

[[inputs.gnmi]]
  addresses = ["arista:6030"]
  encoding = "proto"
  redial = "10s"
  embedded_tags = ["openconfig-interfaces:/interfaces/interface/state/description"]
  
  [[inputs.gnmi.subscription]]
    name = "openconfig-interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/state/counters"
    subscription_mode = "sample"
    sample_interval = "10s"

And even more generically dumping all of the state tree:

[[inputs.gnmi]]
  addresses = ["arista:6030"]
  encoding = "proto"
  redial = "10s"
  embedded_tags = ["openconfig-interfaces:/interfaces/interface/state/description"]
  
  [[inputs.gnmi.subscription]]
    name = "openconfig-interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/state"
    subscription_mode = "sample"
    sample_interval = "10s"

But in neither case am I seeing an added tag:

openconfig-interfaces,host=localhost,name=Management1,source=arista counters/out_octets=0i 1606860676911083197

Have you tried the PR I submitted regarding #7462 to see if it meets your use-case?

https://github.com/influxdata/telegraf/pull/7484

[Tristou27]

Hello @cprecup , @ssoroka and everyone,

Do you have any update about merging this PR?

Thanks!

[jpsalvesen]

Hello everyone,

This is a gentle reminder that merging this PR this will make telegraf a much more useful tool for the telecom industry - increasing the competitiveness of the TICK stack within that domain.

[pederbach]

Hi! Thanks for taking the time to work on this issue!

I'm having some issue getting configuration to work to conform to my use case: Adding interface descriptions as tags to interface counters. I've tried the following two configs:

[[inputs.gnmi]]
  addresses = ["arista:6030"]
  encoding = "proto"
  redial = "10s"
  embedded_tags = ["openconfig-interfaces:/interfaces/interface/state/description"]
  
  [[inputs.gnmi.subscription]]
    name = "openconfig-interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/state/counters"
    subscription_mode = "sample"
    sample_interval = "10s"

And even more generically dumping all of the state tree:

[[inputs.gnmi]]
  addresses = ["arista:6030"]
  encoding = "proto"
  redial = "10s"
  embedded_tags = ["openconfig-interfaces:/interfaces/interface/state/description"]
  
  [[inputs.gnmi.subscription]]
    name = "openconfig-interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/state"
    subscription_mode = "sample"
    sample_interval = "10s"

But in neither case am I seeing an added tag:

openconfig-interfaces,host=localhost,name=Management1,source=arista counters/out_octets=0i 1606860676911083197

Have you tried the PR I submitted regarding #7462 to see if it meets your use-case?

#7484

Did u get it working?

[bewing]

Did u get it working?

I did not, but I am not sure if my issue is PEBCAK, or if because this specific case doesn't apply to openconfig YANG models in my switch platform, as his examples are all Cisco-specific modelings

[cprecup]

Hi! Thanks for taking the time to work on this issue!

I'm having some issue getting configuration to work to conform to my use case: Adding interface descriptions as tags to interface counters. I've tried the following two configs:

[[inputs.gnmi]]
  addresses = ["arista:6030"]
  encoding = "proto"
  redial = "10s"
  embedded_tags = ["openconfig-interfaces:/interfaces/interface/state/description"]
  
  [[inputs.gnmi.subscription]]
    name = "openconfig-interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/state/counters"
    subscription_mode = "sample"
    sample_interval = "10s"

And even more generically dumping all of the state tree:

[[inputs.gnmi]]
  addresses = ["arista:6030"]
  encoding = "proto"
  redial = "10s"
  embedded_tags = ["openconfig-interfaces:/interfaces/interface/state/description"]
  
  [[inputs.gnmi.subscription]]
    name = "openconfig-interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/state"
    subscription_mode = "sample"
    sample_interval = "10s"

But in neither case am I seeing an added tag:

openconfig-interfaces,host=localhost,name=Management1,source=arista counters/out_octets=0i 1606860676911083197

Hello @bewing,

Sorry for the late reply, I didn't have the resources available to work on this. Just checked with this specific OC model on a Cisco box (IOS-XR) and it works on my end.

Here is the config:

[[inputs.cisco_telemetry_gnmi]]
  ## Address and port of the GNMI GRPC server
  addresses = ["x.f.g.h:57777"]

  ## define credentials
  username = "dffg"
  password = "dfgdfg"

  ## GNMI encoding requested (one of: "proto", "json", "json_ietf")
  encoding = "proto"

  ## redial in case of failures after
  redial = "10s"

  embedded_tags = ["openconfig-interfaces:/interfaces/interface/state/description"]

  [[inputs.cisco_telemetry_gnmi.subscription]]
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/state"

    # Subscription mode (one of: "target_defined", "sample", "on_change") and interval
    subscription_mode = "sample"
    sample_interval = "60s"

[[outputs.file]]
  files = ["stdout", "test-oc-if-embedded"]
  data_format = "influx"

I get an output as follows:

state,description=GigabitEth\ 0/0/0/0,host=telegraf,name=GigabitEthernet0/0/0/0,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/0",enabled=false,ifindex=6i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028120000000

state,description=GigabitEth\ 0/0/0/1,host=telegraf,name=GigabitEthernet0/0/0/1,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/1",enabled=false,ifindex=7i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028123000000

state,description=GigabitEth\ 0/0/0/2,host=telegraf,name=GigabitEthernet0/0/0/2,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/2",enabled=true,ifindex=8i,admin_status="UP",oper_status="UP",last_change=145i 1614932028126000000

state,host=telegraf,name=GigabitEthernet0/0/0/3,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/3",enabled=false,ifindex=9i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028129000000

state,host=telegraf,name=GigabitEthernet0/0/0/4,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/4",enabled=false,ifindex=10i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028132000000

state,host=telegraf,name=GigabitEthernet0/0/0/5,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/5",enabled=false,ifindex=11i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028135000000

state,host=telegraf,name=GigabitEthernet0/0/0/6,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/6",enabled=false,ifindex=12i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028138000000

state,host=telegraf,name=Loopback800,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="softwareLoopback",mtu=1500i,name="Loopback800",enabled=true,ifindex=13i,admin_status="UP",oper_status="UP",last_change=1608031449i 1614932028140000000

state,description=Management,host=telegraf,name=MgmtEth0/RP0/CPU0/0,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="MgmtEth0/RP0/CPU0/0",enabled=false,ifindex=3i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028143000000

state,host=telegraf,name=Null0,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="other",mtu=1500i,name="Null0",enabled=true,ifindex=1i,admin_status="UP",oper_status="UP",last_change=1608031366i 1614932028145000000

The level of propagation of an artificial key (embedded key) depends on how the messages are independently encapsulated over gNMI by the box. If they don't carry a description with them, there is no way of telling which description matches with which number of bytes. At a first glance, this may be why there isn't a description label propagated further down to the counters level.

---

On the question:

Have you tried the PR I submitted regarding #7462 to see if it meets your use-case? #7484

Not yet, no. Does it essentially imply creating a separate subscription for each field and comparing strings in the collection of subsequent subscriptions?

[pederbach]

Hi! Thanks for taking the time to work on this issue! I'm having some issue getting configuration to work to conform to my use case: Adding interface descriptions as tags to interface counters. I've tried the following two configs:

[[inputs.gnmi]]
  addresses = ["arista:6030"]
  encoding = "proto"
  redial = "10s"
  embedded_tags = ["openconfig-interfaces:/interfaces/interface/state/description"]
  
  [[inputs.gnmi.subscription]]
    name = "openconfig-interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/state/counters"
    subscription_mode = "sample"
    sample_interval = "10s"

And even more generically dumping all of the state tree:

[[inputs.gnmi]]
  addresses = ["arista:6030"]
  encoding = "proto"
  redial = "10s"
  embedded_tags = ["openconfig-interfaces:/interfaces/interface/state/description"]
  
  [[inputs.gnmi.subscription]]
    name = "openconfig-interfaces"
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/state"
    subscription_mode = "sample"
    sample_interval = "10s"

But in neither case am I seeing an added tag: openconfig-interfaces,host=localhost,name=Management1,source=arista counters/out_octets=0i 1606860676911083197

Hello @bewing,

Sorry for the late reply, I didn't have the resources available to work on this. Just checked with this specific OC model on a Cisco box (IOS-XR) and it works on my end.

Here is the config:

[[inputs.cisco_telemetry_gnmi]]
  ## Address and port of the GNMI GRPC server
  addresses = ["x.f.g.h:57777"]

  ## define credentials
  username = "dffg"
  password = "dfgdfg"

  ## GNMI encoding requested (one of: "proto", "json", "json_ietf")
  encoding = "proto"

  ## redial in case of failures after
  redial = "10s"

  embedded_tags = ["openconfig-interfaces:/interfaces/interface/state/description"]

  [[inputs.cisco_telemetry_gnmi.subscription]]
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/state"

    # Subscription mode (one of: "target_defined", "sample", "on_change") and interval
    subscription_mode = "sample"
    sample_interval = "60s"

[[outputs.file]]
  files = ["stdout", "test-oc-if-embedded"]
  data_format = "influx"

I get an output as follows:

state,description=GigabitEth\ 0/0/0/0,host=telegraf,name=GigabitEthernet0/0/0/0,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/0",enabled=false,ifindex=6i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028120000000

state,description=GigabitEth\ 0/0/0/1,host=telegraf,name=GigabitEthernet0/0/0/1,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/1",enabled=false,ifindex=7i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028123000000

state,description=GigabitEth\ 0/0/0/2,host=telegraf,name=GigabitEthernet0/0/0/2,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/2",enabled=true,ifindex=8i,admin_status="UP",oper_status="UP",last_change=145i 1614932028126000000

state,host=telegraf,name=GigabitEthernet0/0/0/3,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/3",enabled=false,ifindex=9i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028129000000

state,host=telegraf,name=GigabitEthernet0/0/0/4,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/4",enabled=false,ifindex=10i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028132000000

state,host=telegraf,name=GigabitEthernet0/0/0/5,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/5",enabled=false,ifindex=11i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028135000000

state,host=telegraf,name=GigabitEthernet0/0/0/6,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="GigabitEthernet0/0/0/6",enabled=false,ifindex=12i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028138000000

state,host=telegraf,name=Loopback800,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="softwareLoopback",mtu=1500i,name="Loopback800",enabled=true,ifindex=13i,admin_status="UP",oper_status="UP",last_change=1608031449i 1614932028140000000

state,description=Management,host=telegraf,name=MgmtEth0/RP0/CPU0/0,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="ethernetCsmacd",mtu=1514i,name="MgmtEth0/RP0/CPU0/0",enabled=false,ifindex=3i,admin_status="DOWN",oper_status="DOWN",last_change=0i 1614932028143000000

state,host=telegraf,name=Null0,path=openconfig-interfaces:/interfaces/interface/state,source=x.f.g.h type="other",mtu=1500i,name="Null0",enabled=true,ifindex=1i,admin_status="UP",oper_status="UP",last_change=1608031366i 1614932028145000000

The level of propagation of an artificial key (embedded key) depends on how the messages are independently encapsulated over gNMI by the box. If they don't carry a description with them, there is no way of telling which description matches with which number of bytes. At a first glance, this may be why there isn't a description label propagated further down to the counters level.

On the question:

Have you tried the PR I submitted regarding #7462 to see if it meets your use-case? #7484

Not yet, no. Does it essentially imply creating a separate subscription for each field and comparing strings in the collection of subsequent subscriptions?

For policy-map / qos on NCS 5508, we're only getting the data for "class-default" but not the data from other classes - like "MATCH-EF"

When I go into the adv. netconf explorer I can easy get read out the value from MATCH-EF, but we struggle to get it thru gNMI.

[[inputs.gnmi.subscription]] name = "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics" origin = "Cisco-IOS-XR-qos-ma-oper" path = "/qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics/"

subscription_mode = "sample"
sample_interval = "10s"

embedded_tags = ["Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics/class_stats/class_$ "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics/class_stats/cac_st$ "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics/policy_name", "Cisco-IOS-XR-qos-ma-oper:qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics/state"]

Any clue?

Should this work?

[Tristou27]

Hello @pederbach

We were struggling at the same point on QoS XR Yang models. The main issue is that the 'class-name' is not considered as a key and then the embedded_tags feature is solving the issue. The issue is that as class-name is not a key, datas are listed at the same level in the gNMI output. As leaf have the same name, only the last value is retained (class-default).

In the embedded_tags, we've put the 'class-name' leaf. Thanks to that, we retrieved the stats for other classes.

embedded_tags = ["Cisco-IOS-XR-qos-ma-oper:/qos/interface-table/interface/output/service-policy-names/service-policy-instance/statistics/class-stats/class-name",
"Cisco-IOS-XR-qos-ma-oper:/qos/interface-table/interface/input/service-policy-names/service-policy-instance/statistics/class-stats/class-name"]

The embedded_tags must be set at the global gNMI level (just below router addresses for instance) and not under the subscription box. Hope it helps.

[bewing]

Hello @bewing,

Sorry for the late reply, I didn't have the resources available to work on this. Just checked with this specific OC model on a Cisco box (IOS-XR) and it works on my end.

Here is the config: .. The level of propagation of an artificial key (embedded key) depends on how the messages are independently encapsulated over gNMI by the box. If they don't carry a description with them, there is no way of telling which description matches with which number of bytes. At a first glance, this may be why there isn't a description label propagated further down to the counters level.

Thanks for digging into the OC side of things, @cprecup

Unfortunately, this doesn't appear to work with Arista's /bin/OpenConfig or /usr/bin/Octa daemons (Testing on 4.22.5M).

$ git fetch upstream pull/8344/head:embedded_tags
$ git checkout embedded_tags
$ make
$ cat telegraf.conf
[[inputs.cisco_telemetry_gnmi]]
  addresses = ["arista:6030"]

  ## GNMI encoding requested (one of: "proto", "json", "json_ietf")
  encoding = "proto"

  ## redial in case of failures after
  redial = "10s"

  embedded_tags = ["openconfig-interfaces:/interfaces/interface/state/description"]

  [[inputs.cisco_telemetry_gnmi.subscription]]
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/state"

    # Subscription mode (one of: "target_defined", "sample", "on_change") and interval
    subscription_mode = "sample"
    sample_interval = "60s"

$ ./telegraf --config telegraf.conf
...
state,host=hostname,name=Ethernet42,source=arista counters/out_errors=0i 1606860676908483220
state,host=hostname,name=Ethernet42,source=arista counters/out_multicast_pkts=0i 1606860676908475571
state,host=hostname,name=Ethernet42,source=arista counters/out_octets=0i 1606860676908537087
state,host=hostname,name=Ethernet42,source=arista counters/out_unicast_pkts=0i 1606860676908501431
state,host=hostname,name=Ethernet42,source=arista description="stuff" 1606860677084140118

On the question:

Have you tried the PR I submitted regarding #7462 to see if it meets your use-case? #7484

Not yet, no. Does it essentially imply creating a separate subscription for each field and comparing strings in the collection of subsequent subscriptions?

It does. In my testing, we use on_change subscriptions for tags (description, BGP neighbor name, etc) and store them in-memory, and during metric processing it searches for tags to apply. I have not tested it at scale, as my current planned deployment model is per-switch agents, not a centralized poller. I haven't looked at it in 6 months other than the occasional rebase, however, so I might be a bit fuzzy on it.

[cprecup]

Hello @bewing,

Wondering what you see for the rest of the metrics under state. I'd expect that the description tag is attached to these, as in my previous message.

Thanks for digging into the OC side of things, @cprecup

Unfortunately, this doesn't appear to work with Arista's /bin/OpenConfig or /usr/bin/Octa daemons (Testing on 4.22.5M).

$ git fetch upstream pull/8344/head:embedded_tags
$ git checkout embedded_tags
$ make
$ cat telegraf.conf
[[inputs.cisco_telemetry_gnmi]]
  addresses = ["arista:6030"]

  ## GNMI encoding requested (one of: "proto", "json", "json_ietf")
  encoding = "proto"

  ## redial in case of failures after
  redial = "10s"

  embedded_tags = ["openconfig-interfaces:/interfaces/interface/state/description"]

  [[inputs.cisco_telemetry_gnmi.subscription]]
    origin = "openconfig-interfaces"
    path = "/interfaces/interface/state"

    # Subscription mode (one of: "target_defined", "sample", "on_change") and interval
    subscription_mode = "sample"
    sample_interval = "60s"

$ ./telegraf --config telegraf.conf
...
state,host=hostname,name=Ethernet42,source=arista counters/out_errors=0i 1606860676908483220
state,host=hostname,name=Ethernet42,source=arista counters/out_multicast_pkts=0i 1606860676908475571
state,host=hostname,name=Ethernet42,source=arista counters/out_octets=0i 1606860676908537087
state,host=hostname,name=Ethernet42,source=arista counters/out_unicast_pkts=0i 1606860676908501431
state,host=hostname,name=Ethernet42,source=arista description="stuff" 1606860677084140118

The level of propagation of an artificial key (embedded key) depends on how the messages are independently encapsulated over gNMI by the box. If they don't carry a description with them, there is no way of telling which description matches with which number of bytes. At a first glance, this may be why there isn't a description label propagated further down to the counters level.

As per what is written above, I've debugged the content of the messages (Update). The state ones come in a different Update message than the counters ones, and hence counters don't contain the description in their message:

State fields:

Counters:

I'd suspect this is the behaviour that you are experiencing and it looks like it is dependent on the source of the streaming.

Testing environment: Cisco IOS-XR.

On the on_change-based approach from https://github.com/influxdata/telegraf/pull/7484:

Please check https://github.com/influxdata/telegraf/pull/7484#issuecomment-802679092.

[bewing]

Hello @bewing,

Wondering what you see for the rest of the metrics under state. I'd expect that the description tag is attached to these, as in my previous message.

I'm strongly suspecting this is a result in the differences in the GRPC implementations of IOX-XR and Arista.

I did a packet capture of the GRPC exchange -- the gnmi.SubscribeRequest I send doesn't mention interface descriptions at all, so Arista doesn't send them:

All the Arista sends after subscribing is just counter data -- no extra tags:

Even if I change my subscription to /interfaces/interface/state, so that I get the descriptions as part of an update message, it doesn't make it into the emitted telegraf metric, possibly because of how metrics are grouped into responses by Arista:

I tried to spin up an AWS EC2 instance of XRv 6.3 from the marketplace instance to test more, but I can't SSH to it, so I unfortunately can't compare behavior between the two platforms with the same configuration. :/

[slampt]

@bewing what version of EOS are you playing with? I am currently looking into this and having the description as a tag would be very handy when it comes to graphing specific things in Grafana

Here is an output from EOS 4.23 for a subinterface

--path interfaces/interface[name=Ethernet1/1]/subinterfaces/subinterface[index=600]/state Get Response: [ { "time": "1970-01-01T08:00:00+08:00", "updates": [ { "Path": "interfaces/interface[name=Ethernet1/1]/subinterfaces/subinterface[index=600]/state", "values": { "interfaces/interface/subinterfaces/subinterface/state": { "openconfig-interfaces:admin-status": "DOWN", "openconfig-interfaces:counters": { "in-broadcast-pkts": "0", "in-discards": "0", "in-errors": "0", "in-multicast-pkts": "0", "in-octets": "20768202392", "in-unicast-pkts": "18875338", "out-broadcast-pkts": "0", "out-discards": "0", "out-errors": "0", "out-multicast-pkts": "0", "out-octets": "11780243010", "out-unicast-pkts": "79801116" }, "openconfig-interfaces:description": ""Temp sub-int link"", "openconfig-interfaces:enabled": true, "openconfig-interfaces:index": 600, "openconfig-interfaces:name": "Ethernet1/1.600", "openconfig-interfaces:oper-status": "UP" }

Here is 4.26

Get Response: [ { "time": "1970-01-01T08:00:00+08:00", "updates": [ { "Path": "interfaces/interface[name=Ethernet1/1]/subinterfaces/subinterface[index=600]/state", "values": { "interfaces/interface/subinterfaces/subinterface/state": { "openconfig-interfaces:admin-status": "UP", "openconfig-interfaces:counters": { "in-broadcast-pkts": "0", "in-discards": "0", "in-errors": "0", "in-multicast-pkts": "0", "in-octets": "11657940", "in-unicast-pkts": "83271", "out-broadcast-pkts": "0", "out-discards": "0", "out-errors": "0", "out-multicast-pkts": "0", "out-octets": "0", "out-unicast-pkts": "0" }, "openconfig-interfaces:description": ""Temp sub-int link"", "openconfig-interfaces:enabled": true, "openconfig-interfaces:index": 600, "openconfig-interfaces:name": "Ethernet1/1.600", "openconfig-interfaces:oper-status": "UP" } } } ] } ]

Pretty much the same, but unrelated openconfig-interfaces:admin-status seems to return a correct result in 4.26 but not 4.23 Additionally when querying index 0 on 4.23 counters are returned basically matching that of interfaces/interface[name=Ethernet1/1]/state, however in 4.26 it returns empty. I prefer the latter, unfortunately none of this really helps with the Description tag. Also interfaces/interface[name=*]/state is the same.

I am currently collecting counters using the following and dumping into PROM:

[[inputs.gnmi.subscription]] ## Name of the measurement name = "interface" origin = "openconfig-interfaces" path = "/interfaces/interface[name=*]/state" subscription_mode = "sample" sample_interval = "10s"

[[inputs.gnmi.subscription]] ## Name of the measurement name = "subinterface" origin = "openconfig-interfaces" path = "/interfaces/interface[name=]/subinterfaces/subinterface[index=]/state" subscription_mode = "sample" sample_interval = "10s"

Turns out gnmic subscribe dumps the description out once as part of the initial(first?) response, then never again. :(

Let me know if there is anything I can do to help! :)

[bofh16]

Hello, can you share the roadmap for embedded_tags being implemented for inputs.gnmi plugins, please? I got a message it would be included in the 08.03 nightly build, but it doesn't seem to be there yet.

    embedded_tags = [

            "Cisco-IOS-XR-asr9k-np-oper:hardware-module-np/nodes/node/nps/np/counters/np-counter/counter-index",
            "Cisco-IOS-XR-asr9k-np-oper:hardware-module-np/nodes/node/nps/np/counters/np-counter/counter-type",
            "Cisco-IOS-XR-asr9k-np-oper:hardware-module-np/nodes/node/nps/np/counters/np-counter/counter-name",
            "Cisco-IOS-XR-asr9k-fsi-oper:fabric-stats/nodes/node/statses/stats/stats-table/fsi-stat/counter-name"
    ]

    [[inputs.gnmi.subscription]]
            name = "xr64_np_counters"
            origin = "Cisco-IOS-XR-asr9k-np-oper"
            path = "/hardware-module-np/nodes/node/nps/np/counters/np-counter"
            subscription_mode = "sample"
            sample_interval = "10s"

`$ telegraf --config telegraf.conf 2022-03-25T13:04:20Z W! DeprecationWarning: Option "precision" of plugin "outputs.influxdb" deprecated since version 1.0.0 and will be removed in 2.0.0: option is ignored 2022-03-25T13:04:20Z E! [telegraf] Error running agent: Error loading config file telegraf.conf: plugin inputs.gnmi: line 54: configuration specified the fields ["embedded_tags"], but they weren't used

$ telegraf --version Telegraf 1.22.0 (git: HEAD 80a7feb8)`

TIA

[cprecup]

Hello,

@bofh16, afaik, there was a development in parallel to this PR (https://github.com/influxdata/telegraf/pull/7484) which has been merged with the main branch. Not sure where we stand with the current PR as it has been ready to be merged a while back.

[bofh16]

I'm a bit confused, as #7484 was mentioned to me as a resolution for including embeded_tags in inputs.gnmi plugin. This is definitely not the case. Should I consider this PR as a proper solution for this feature and one day, once merged, to have the functionality in place?

[cprecup]

I have a similar level of understanding, @bofh16.

@ssoroka - any thoughts on this topic?

[bewing]

FWIW, I believe this might come down to differences in vendor gNMI server implementations. In all my testing Arista's Octa gNMI daemon emits all individual leaf nodes subscribed to (regardless of the number of subscriptions) as separate gnmi.SubscriptionResponse messages. The approach in #7484 was to store various data in memory for application to readings from subsequent SR messages. Since an individual SR message never contained both a counter reading, and a description tag, the approach in this PR never worked in my Arista testing.

I also believe the approach in #7484 was tested by IOS-XR users, and it unfortunately did not work for them. I do not have access to an IOS-XR device in my current position, and wasn't able to successfully spin up an AWS image, and wasn't able to independently test.

Is it possible for both approaches to exist side by side in the code? Or do we want to think further about how this should be handled?

[srebhan]

@cprecup can you please resolve the merge conflicts of this PR? Furthermore, can you please outline if and how this PR conflicts with #7484? I'm not an expert when it comes to gNMI, so it would be good in which use-case you need the features of this PR and for which #7484 is suited.

@bewing, @bofh16 I'm working on getting this in, would appreciate to get your testing!

[bofh16]

@srebhan I can test it, once available.

[srebhan]

@bofh16 thanks! @cprecup waiting for you feedback...

[cprecup]

@cprecup can you please resolve the merge conflicts of this PR? Furthermore, can you please outline if and how this PR conflicts with #7484? I'm not an expert when it comes to gNMI, so it would be good in which use-case you need the features of this PR and for which #7484 is suited.

@bewing, @bofh16 I'm working on getting this in, would appreciate to get your testing!

Hi @srebhan! Thank you for your message.

I've resolved the conflicts and it requires testing.

As I understand, both implementation address the same issue, missing tags and converting fields to tags in order to distinguish data points that otherwise have identical keys and timestamps.

The difference is in the way this data is collected and how the name mapping is done:

• this PR creates a subscription and checks if the data sent over gNMI in a gnmi.SubscriptionResponse contains the embedded tag as a field name. If yes, then it converts it to a tag before propagation. Stores in memory only the names of the tags and not the values.
• the PR #7484, afaik, creates additional subscriptions for the wanted tags and does a lookup on a set of maps for the right tag entry given a path.

I would use the first version when the gnmi.SubscriptionResponse contains more than one field (this is typically the case with the tested devices used before this PR). Seems like PR #7484 covers the use case in which the device sends separate gnmi.SubscriptionResponse messages, for each field.

Hope that clarifies things.

[telegraf-tiger[bot]]

Hello! I am closing this issue due to inactivity. I hope you were able to resolve your problem, if not please try posting this question in our Community Slack or Community Page. Thank you!

[Tristou27]

Tentative to re-open this PR... we would like to have this embedded_tags feature within gnmi plugin it will definitely solve many issues with IOS-XR.