snmp_exporter icon indicating copy to clipboard operation
snmp_exporter copied to clipboard

Polling fortigate for fgvpn stats returned errors...

Open paulinster opened this issue 4 years ago • 18 comments

Hi Everyone,

Not sure if I am at the right place for this type of question, but if not forgive me...

I have fortigate device running version 6.2.3. I can successfully poll the device, but as soon as I am adding the fgVpnTunTable(.1.3.6.1.4.1.12356.101.12.2.2) but it does fail with error like this...

800 error(s) occurred:
* collected metric "fgVpnTunEntLocGwyIp" { label:<name:"fgVpnTunEntIndex" value:"140" > label:<name:"fgVpnTunEntLocGwyIp" value:"208.94.108.2" > gauge:<value:1 > } was collected before with the same name and label values
* collected metric "fgVpnTunEntSelectorDstEndIp" { label:<name:"fgVpnTunEntIndex" value:"134" > label:<name:"fgVpnTunEntSelectorDstEndIp" value:"10.248.32.255" > gauge:<value:1 > } was collected before with the same name and label values
* collected metric "fgVpnTunEntSelectorSrcBeginIp" { label:<name:"fgVpnTunEntIndex" value:"24" > label:<name:"fgVpnTunEntSelectorSrcBeginIp" value:"192.168.147.0" > gauge:<value:1 > } was collected before with the same name and label values
* collected metric "fgVpnTunEntInOctets" { label:<name:"fgVpnTunEntIndex" value:"136" > counter:<value:1.9405667848e+10 > } was collected before with the same name and label values
* collected metric "fgVpnTunEntTimeout" { label:<name:"fgVpnTunEntIndex" value:"171" > gauge:<value:41053 > } was collected before with the same name and label values
* collected metric "fgVpnTunEntLocGwyIp" { label:<name:"fgVpnTunEntIndex" value:"136" > label:<name:"fgVpnTunEntLocGwyIp" value:"208.94.108.2" > gauge:<value:1 > } was collected before with the same name and label values
###### CUT - Remaining line remove ######  

After digging a bit more I found that it look like the fgVpnTunEntIndex (1.3.6.1.4.1.12356.101.12.2.2.1.1) isn't returned by the Fortigate. I have open a case with them and they say that:

"as per current design (6.2.3 and above) for SNMP query for VPN tunnel list, the phase1 serial and phase2 serial both are used to be indexes for VPN Tunnel list.
So the original fgVpnTunEntIndex is NOT used now"

The fgVpnTunEntIndex became meaningless (just for MIB compatibility). And also, any index of the table should be defined as "not-accessible"."

So I am now confused on how I can query a VPN status or bandwitdh if I can't refer to an index. As per they say and my understanding the only value

Thanx for you help!

paulinster avatar Feb 08 '21 17:02 paulinster

That is very confusing. It seems like they changed the table without implementing the correct changes to the MIB.

SuperQ avatar Feb 08 '21 17:02 SuperQ

@SuperQ indeed they change the table... I have a ticket open with them, but I don't know enought about snmp/RFC to say if it's compliant or not. Their MIB does state that fgVpnTunEntIndex should be present, but in fact it not present/not use anymore.

In another comment from Fortinet they say....

The Old OID 1.3.6.1.4.1.12356.101.12.2.2.1.20.6 needs to be changed to the New OID 1.3.6.1.4.1.12356.101.12.2.2.1.20.6.x if you use snmpget, where x might be any of the p2serials which has the same p1serial as 6. In a more precise way, the OID is not changed, which is 1.3.6.1.4.1.12356.101.12.2.2.1.20 (fgVpnTunEntStatus). The last number(s) is/are just the indexes for this table item. That means, when you do "snmpwalk -v2c -cpublic host 1.3.6.1.4.1.12356.101.12.2.2.1.20", you will get all fgVpnTunEntStatus indexed by (p1serial, p2serial),since one phase1 could be used by multiple phase2. This is an improvement for the SNMP querying fgVpnTunTable.

However I am confused on how I can retrieve/refer to the proper value without an index...

paulinster avatar Feb 08 '21 17:02 paulinster

Yes, that's definitely a MIB INDEX change. You can't just add an additional index entry without updating the MIB, it changes the depth of the tree.

SuperQ avatar Feb 08 '21 18:02 SuperQ

Would you be able to post a sample snmpwalk of the table? I only need one full entry of something like 1.3.6.1.4.1.12356.101.12.2.2.1.x.y.z.0. I think that's how deep the tree should go based on the support response.

SuperQ avatar Feb 08 '21 18:02 SuperQ

I wonder if this is the correct patch to apply to the MIB:

diff --git a/Fortigate/FORTINET-FORTIGATE-MIB.mib b/Fortigate/FORTINET-FORTIGATE-MIB.mib
index 95a240c..798b114 100644
--- a/Fortigate/FORTINET-FORTIGATE-MIB.mib
+++ b/Fortigate/FORTINET-FORTIGATE-MIB.mib
@@ -4899,7 +4899,7 @@ fgVpnTunEntry OBJECT-TYPE
     STATUS      current
     DESCRIPTION 
         "Tunnel VPN peer info"
-    INDEX       { fgVpnTunEntIndex }
+    INDEX       { fgVpnTunEntIndex, fgVpnTunEntVdom }
     ::= { fgVpnTunTable 1 }
 
 FgVpnTunEntry ::= SEQUENCE {

The only way to really tell is to match up the values of those two indexes with the index results.

SuperQ avatar Feb 08 '21 19:02 SuperQ

Here's an snmpwalk out of fgVpnTunEntIndex, fgVpnTunEntPhase1Name, fgVpnTunEntPhase2Name and fgVpnTunEntStatus

As you'll see at fgVpnTunEntIndex, no result is return....

fgVpnTunEntIndex(1.3.6.1.4.1.12356.101.12.2.2.1.1)

lpaulin@MTL625QJHD2-MAC:~/workdir$ snmpbulkwalk -v2c -c stingrayread -Cr25 10.1.0.81 1.3.6.1.4.1.12356.101.12.2.2.1.1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.1 = No Such Instance currently exists at this OID

fgVpnTunEntPhase1Name(1.3.6.1.4.1.12356.101.12.2.2.1.2)

lpaulin@MTL625QJHD2-MAC:~/workdir$ snmpbulkwalk -v2c -c stingrayread -Cr25 10.1.0.81 1.3.6.1.4.1.12356.101.12.2.2.1.2 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.180.52 = STRING: "RO_NOVRA" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.180.53 = STRING: "RO_NOVRA" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.180.54 = STRING: "RO_NOVRA" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.180.55 = STRING: "RO_NOVRA" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.180.56 = STRING: "RO_NOVRA" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.180.57 = STRING: "RO_NOVRA" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.180.58 = STRING: "RO_NOVRA" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.180.59 = STRING: "RO_NOVRA" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.180.61 = STRING: "RO_NOVRA" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.180.63 = STRING: "RO_NOVRA"

fgVpnTunEntPhase2Name(1.3.6.1.4.1.12356.101.12.2.2.1.3)

lpaulin@MTL625QJHD2-MAC:~/workdir$ snmpbulkwalk -v2c -c stingrayread -Cr25 10.1.0.81 1.3.6.1.4.1.12356.101.12.2.2.1.3 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.180.52 = STRING: "RO_NOVRA_P1" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.180.53 = STRING: "RO_NOVRA_P2" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.180.54 = STRING: "RO_NOVRA_P3" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.180.55 = STRING: "RO_NOVRA_P4" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.180.56 = STRING: "RO_NOVRA_P5" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.180.57 = STRING: "RO_NOVRA_P6" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.180.58 = STRING: "RO_NOVRA_P7" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.180.59 = STRING: "RO_NOVRA_P8" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.180.61 = STRING: "RO_NOVRA_P9" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.180.63 = STRING: "RO_NOVRA_P10"

fgVpnTunEntStatus(1.3.6.1.4.1.12356.101.12.2.2.1.20)

lpaulin@MTL625QJHD2-MAC:~/workdir$ snmpbulkwalk -v2c -c stingrayread -Cr25 10.1.0.81 1.3.6.1.4.1.12356.101.12.2.2.1.20 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.180.52 = INTEGER: 2 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.180.53 = INTEGER: 2 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.180.54 = INTEGER: 2 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.180.55 = INTEGER: 2 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.180.56 = INTEGER: 2 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.180.57 = INTEGER: 2 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.180.58 = INTEGER: 2 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.180.59 = INTEGER: 2 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.180.61 = INTEGER: 2 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.180.63 = INTEGER: 1

paulinster avatar Feb 08 '21 19:02 paulinster

What about fgVpnTunEntVdom, 1.3.6.1.4.1.12356.101.12.2.2.1.21?

SuperQ avatar Feb 08 '21 19:02 SuperQ

Here you go for fgVpnTunEntVdom

lpaulin@MTL625QJHD2-MAC:~$ snmpbulkwalk -v2c -c stingrayread -Cr25 10.1.0.81 .1.3.6.1.4.1.12356.101.12.2.2.1.21 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.180.52 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.180.53 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.180.54 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.180.55 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.180.56 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.180.57 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.180.58 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.180.59 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.180.61 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.180.63 = INTEGER: 1

paulinster avatar Feb 08 '21 20:02 paulinster

Hmm, that's not an index then.

I guess, this is hard to fix, because the MIB doesn't reflect the reality of their walk results.

We don't really have a trick in the generator to override the indexes for this.

SuperQ avatar Feb 08 '21 20:02 SuperQ

The fact is that if I run the same command on previous Fortigate version, ie 6.0.4, it does return the index and the output is different, see above for and output

If this can be of any help, the following output were perform on a device running Fortios 6.04

fgVpnTunEntIndex(1.3.6.1.4.1.12356.101.12.2.2.1.1) SNMPv2-SMI::enterprises.12356.101.12.2.2.1.1.1 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.1.2 = INTEGER: 2 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.1.3 = INTEGER: 3 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.1.4 = INTEGER: 4 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.1.5 = INTEGER: 5

fgVpnTunEntVdom(1.3.6.1.4.1.12356.101.12.2.2.1.21) SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.1 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.2 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.3 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.4 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.21.5 = INTEGER: 1

fgVpnTunEntStatus(1.3.6.1.4.1.12356.101.12.2.2.1.20) SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.1 = INTEGER: 2 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.2 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.3 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.4 = INTEGER: 1 SNMPv2-SMI::enterprises.12356.101.12.2.2.1.20.5 = INTEGER: 1

fgVpnTunEntPhase1Name(1.3.6.1.4.1.12356.101.12.2.2.1.2) SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.1 = STRING: "RO_CLX" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.2 = STRING: "RO_CLX" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.3 = STRING: "RO_CLX" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.4 = STRING: "RO_CLX" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.2.5 = STRING: "RO_CLX"

fgVpnTunEntPhase2Name(1.3.6.1.4.1.12356.101.12.2.2.1.3) SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.1 = STRING: "RO_CLX_P2-1" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.2 = STRING: "RO_CLX_P2-2" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.3 = STRING: "RO_CLX_P2-3" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.4 = STRING: "RO_CLX_P2-4" SNMPv2-SMI::enterprises.12356.101.12.2.2.1.3.5 = STRING: "RO_CLX_P2-5"

Am I wrong saying that what they did/change totally break the RFC standard so object in a table should refer to an index entry

paulinster avatar Feb 08 '21 21:02 paulinster

Yes, I'm 99% sure they are not following MIB standards here. They changed from a single octet index to a two octet index. The MIB doesn't match anymore.

SuperQ avatar Feb 08 '21 22:02 SuperQ

Great Thanx for your time.. If you find a workaround, let me know, but in meantime I'll update the ticket with Fortinet

paulinster avatar Feb 09 '21 14:02 paulinster

The only workaround I could come up with was to hand-craft a custom snmp.yml. For example, here's one entry to pattern after. I have no idea what the correct label names should be, since they haven't updated the MIB to reflect the new table structure.

fortinet:
  walk:
  - 1.3.6.1.4.1.12356.101.12.2.2.1
  metrics:
  - name: fgVpnTunEntPhase1Name
    oid: 1.3.6.1.4.1.12356.101.12.2.2.1.2
    type: DisplayString
    help: Descriptive name of phase1 configuration for the tunnel - 1.3.6.1.4.1.12356.101.12.2.2.1.2
    indexes:
    - labelname: phase1serial
      type: gauge
    - labelname: phase2serial
      type: gauge

SuperQ avatar Feb 09 '21 15:02 SuperQ

The only workaround I could come up with was to hand-craft a custom snmp.yml. For example, here's one entry to pattern after. I have no idea what the correct label names should be, since they haven't updated the MIB to reflect the new table structure.

fortinet:
  walk:
  - 1.3.6.1.4.1.12356.101.12.2.2.1
  metrics:
  - name: fgVpnTunEntPhase1Name
    oid: 1.3.6.1.4.1.12356.101.12.2.2.1.2
    type: DisplayString
    help: Descriptive name of phase1 configuration for the tunnel - 1.3.6.1.4.1.12356.101.12.2.2.1.2
    indexes:
    - labelname: phase1serial
      type: gauge
    - labelname: phase2serial
      type: gauge

Hi i have the same problem. I did this change on fgVpnTunEntPhase1Name but still has the error.

mariano-tuentrada avatar Mar 18 '21 23:03 mariano-tuentrada

I have forgot to update but the issue got confirmation by the vendor couple of weeks ago that they will solve the issue unnder bud id 0696836. The issue should be resolve in the newest coming release, Fortios 7.0 and also in version version 6.2.9 . However I don't know about 6.4 releases

paulinster avatar Mar 22 '21 01:03 paulinster

Fortinet released an updated MIB with 7.0 where they addressed the issue.

lubyou avatar Mar 31 '21 07:03 lubyou

If only they would publish the MIBs somewhere public.

SuperQ avatar Mar 31 '21 07:03 SuperQ

Hey guys. In my repository find a latest versión 7.0.1 mibs, https://github.com/Delfos98/Delfos98.git I'm create by only this case.

Delfos98 avatar Aug 04 '21 16:08 Delfos98