Flow collector issue with templates

[xmacan]

From forum, actual develop branch:

[15-May-2024 09:55:44 America/New_York] PHP Fatal error: Uncaught TypeError: count(): Argument #1 ($value) must be of type Countable|array, false given in /var/www/html/cacti/plugins/flowview/flowview_process.php:102 Stack trace: #0 {main} thrown in /var/www/html/cacti/plugins/flowview/flowview_process.php on line 102

Additionally, in the cacti logs I see the flowview plugin trying to constantly create tables in the database since they "don't exist", but i can confirm in mysql i am seeing the following:

plugin_flowview_devices plugin_flowview_dnscache plugin_flowview_ports plugin_flowview_queries plugin_flowview_schedules

Finally, this also shows up in the cacti log (GUI) after a minute or so after enabling the plugin.

CMDPHP ERROR: A DB Row Failed!, Error: Unknown column 'enabled' in 'where clause'

[arno-st]

Maybe the line 544 has to be split up a little bit, to give more information on what can be the problem:

if ($connected && !flowview_db_table_exists('%flowview%') && api_plugin_is_enabled('flowview')) {
			cacti_log('Setting Up Database Tables Since they do not exist', false, 'FLOWVIEW');

Doing all test in one line didn't give much of it.

As for the error in flowview_process.php, if there is a problem to create/access the table in the setup process, I doubt other line will be able to access it. So the first will be to chek why it's always trying to create the table, then I'm sure it will solve the flowview_process.php issue.

[TheWitness]

Sorry all, I was hoping to help on this. Life is complicated.

[xmacan]

No problem, I will try it and let you know if I need help

[TheWitness]

Things should be better now guys. Test and report back.

[xmacan]

new errors: 2024-05-18 16:14:53 - CMDPHP PHP ERROR WARNING Backtrace: (/plugins/flowview/flow_collector.php[347]:process_fv9(), /plugins/flowview/flow_collector.php[522]:CactiErrorHandler())

2024-05-18 16:14:53 - ERROR PHP WARNING in Plugin 'flowview': Trying to access array offset on false in file: /usr/local/share/cacti/plugins/flowview/flow_collector.php on line: 522 2024-05-18 16:14:53 - CMDPHP PHP ERROR WARNING Backtrace: (/plugins/flowview/flow_collector.php[347]:process_fv9(), /plugins/flowview/flow_collector.php[521]:CactiErrorHandler()) 2024-05-18 16:14:53 - ERROR PHP WARNING in Plugin 'flowview': Trying to access array offset on false in file: /usr/local/share/cacti/plugins/flowview/flow_collector.php on line: 521 2024-05-18 16:14:53 - CMDPHP PHP ERROR WARNING Backtrace: (/plugins/flowview/flow_collector.php[347]:process_fv9(), /plugins/flowview/flow_collector.php[519]:unpack(), CactiErrorHandler()) 2024-05-18 16:14:53 - ERROR PHP WARNING in Plugin 'flowview': unpack(): Type n: not enough input values, need 2 values but only 0 were provided in file: /usr/local/share/cacti/plugins/flowview/flow_collector.php on line: 519

[xmacan]

debug data: Flow: Sleep Time: 0.00 Flow: Packet from: 192.168.222.1:3952 v9 - Len: 1280 Flow: Processing v9 Data, Records: 15 Flow: Data Found, Awaiting Templates Flow: Cycle Time: 0.00

Flow: Sleep Time: 0.00 Flow: Packet from: 192.168.222.1:3952 v9 - Len: 1140 Flow: Processing v9 Data, Records: 3 Flow: Template Found Flow: Template Len: 1044 Flow: Template Id: 258 with 19 fields Flow: Template Id: 260 with 17 fields Flow: Template Id: 262 with 23 fields Flow: Template Id: 266 with 21 fields Flow: Template Id: 263 with 23 fields Flow: Template Id: 267 with 21 fields Flow: Template Id: 259 with 19 fields Flow: Template Id: 261 with 17 fields Flow: Template Id: 264 with 23 fields Flow: Template Id: 268 with 21 fields Flow: Template Id: 265 with 23 fields Flow: Template Id: 269 with 21 fields Flow: Templates Captured PHP Warning: unpack(): Type n: not enough input values, need 2 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 519

Warning: unpack(): Type n: not enough input values, need 2 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 519 PHP Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 521

Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 521 PHP Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 522

Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 522 Flow: Cycle Time: 0.00

[TheWitness]

Oh, it's at the end of the template...

[TheWitness]

Likely it should break. Maybe the length calculation needs to be checked or a break needs to be set at the end of the buffer.

[xmacan]

new errors: 2024-05-19 17:41:20 - CMDPHP PHP ERROR WARNING Backtrace: (/plugins/flowview/flow_collector.php[356]:process_fv9(), /plugins/flowview/flow_collector.php[663]:unpack(), CactiErrorHandler())

2024-05-19 17:41:20 - ERROR PHP WARNING in Plugin 'flowview': unpack(): Type N: not enough input values, need 4 values but only 1 was provided in file: /usr/local/share/cacti/plugins/flowview/flow_collector.php on line: 663 2024-05-19 17:41:20 - CMDPHP PHP ERROR WARNING Backtrace: (/plugins/flowview/flow_collector.php[356]:process_fv9(), /plugins/flowview/flow_collector.php[713]:process_v9_v10(), /plugins/flowview/flow_collector.php[1072]:CactiErrorHandler()) 2024-05-19 17:41:20 - ERROR PHP WARNING in Plugin 'flowview': Undefined array key -1123061029 in file: /usr/local/share/cacti/plugins/flowview/flow_collector.php on line: 1072 2024-05-19 17:41:20 - CMDPHP PHP ERROR WARNING Backtrace: (/plugins/flowview/flow_collector.php[356]:process_fv9(), /plugins/flowview/flow_collector.php[713]:process_v9_v10(), /plugins/flowview/flow_collector.php[1072]:CactiErrorHandler()) 2024-05-19 17:41:20 - ERROR PHP WARNING in Plugin 'flowview': Undefined array key -1123061029 in file: /usr/local/share/cacti/plugins/flowview/flow_collector.php on line: 1072 2024-05-19 17:41:20 - CMDPHP PHP ERROR WARNING Backtrace: (/plugins/flowview/flow_collector.php[356]:process_fv9(), /plugins/flowview/flow_collector.php[713]:process_v9_v10(), /plugins/flowview/flow_collector.php[1072]:CactiErrorHandler()) 2024-05-19 17:41:20 - ERROR PHP WARNING in Plugin 'flowview': Undefined array key -1123061029 in file: /usr/local/share/cacti/plugins/flowview/flow_collector.php on line: 1072

[TheWitness]

This is fixed now.

[xmacan]

@TheWitness I thing, that it isn't fixed. Actual develop branch and still: 2024-05-20 15:10:01 - ERROR PHP WARNING in Plugin 'flowview': Trying to access array offset on null in file: /usr/local/share/cacti/plugins/flowview/functions.php on line: 3613 2024-05-20 15:10:01 - CMDPHP PHP ERROR WARNING Backtrace: (/plugins/flowview/flow_collector.php[356]:process_fv9(), /plugins/flowview/flow_collector.php[713]:process_v9_v10(), /plugins/flowview/flow_collector.php[1086]:flowview_get_dns_from_ip(), /plugins/flowview/functions.php[3348]:flowview_get_owner_from_arin(), /plugins/flowview/functions.php[3613]:CactiErrorHandler()) 2024-05-20 15:10:01 - ERROR PHP DEPRECATED in Plugin 'flowview': strtolower(): Passing null to parameter #1 ($string) of type string is deprecated in file: /usr/local/share/cacti/plugins/flowview/functions.php on line: 3613 2024-05-20 15:10:01 - CMDPHP PHP ERROR Backtrace: (/plugins/flowview/flow_collector.php[356]:process_fv9(), /plugins/flowview/flow_collector.php[713]:process_v9_v10(), /plugins/flowview/flow_collector.php[1086]:flowview_get_dns_from_ip(), /plugins/flowview/functions.php[3348]:flowview_get_owner_from_arin(), /plugins/flowview/functions.php[3613]:strtolower(), CactiErrorHandler()) 2024-05-20 15:10:01 - ERROR PHP WARNING in Plugin 'flowview': unpack(): Type N: not enough input values, need 4 values but only 1 was provided in file: /usr/local/share/cacti/plugins/flowview/flow_collector.php on line: 663 2024-05-20 15:10:01 - CMDPHP PHP ERROR WARNING Backtrace: (/plugins/flowview/flow_collector.php[356]:process_fv9(), /plugins/flowview/flow_collector.php[663]:unpack(), CactiErrorHandler()) 2024-05-20 15:10:01 - ERROR PHP ERROR in Plugin 'flowview': Uncaught TypeError: count(): Argument #1 ($value) must be of type Countable|array, false given in /usr/local/share/cacti/plugins/flowview/flow_collector.php:687 Stack trace: #0 /usr/local/share/cacti/plugins/flowview/flow_collector.php(356): process_fv9('\x00\t\x00\x01G\x8CUhfKK\xA7\x00\t\xA6...', '192.168.222.1:3...') #1 {main} thrown in file: /usr/local/share/cacti/plugins/flowview/flow_collector.php on line: 687 2024-05-20 15:10:01 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler())

[TheWitness]

That another issue entirely. We need to audit the ARIN algo.

[TheWitness]

I would not be surprised is they have switched things up.

[xmacan]

Flow: Template Found: 262 Flow: Processing Record, Remaining: 1 Flow: Processing Record, Remaining: -78 Flow: Writing Records: 19 Flow: Cycle Time: 0.52

Flow: Sleep Time: 0.00 Flow: Packet from: 192.168.222.1:3952 v9 - Len: 1344 Flow: Processing v9 Data, Records: 19 Flow: Processing v9 Data, Record: 0, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 1, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 2, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 3, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 4, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 5, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 6, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 7, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 8, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 9, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 10, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 11, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 12, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 13, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 14, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 15, FSId:260, FSLen:68 Flow: Data Found, Processing Flow: Template Found: 260 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 16, FSId:262, FSLen:84 Flow: Data Found, Processing Flow: Template Found: 262 Flow: Processing Record, Remaining: 1 Flow: Processing Record, Remaining: -78 Flow: Processing v9 Data, Record: 18, FSId:262, FSLen:84 Flow: Data Found, Processing Flow: Template Found: 262 Flow: Processing Record, Remaining: 1 PHP Warning: unpack(): Type C: not enough input values, need 1 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 663

Warning: unpack(): Type C: not enough input values, need 1 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 663 PHP Fatal error: Uncaught TypeError: implode(): Argument #2 ($array) must be of type ?array, false given in /usr/local/share/cacti/plugins/flowview/flow_collector.php:666 Stack trace: #0 /usr/local/share/cacti/plugins/flowview/flow_collector.php(666): implode('.', false) #1 /usr/local/share/cacti/plugins/flowview/flow_collector.php(356): process_fv9('\x00\t\x00\x13H\xAF\rxfK\x96\x13\x00\n\xCA...', '192.168.222.1:3...') #2 {main} thrown in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 666

Fatal error: Uncaught TypeError: implode(): Argument #2 ($array) must be of type ?array, false given in /usr/local/share/cacti/plugins/flowview/flow_collector.php:666 Stack trace: #0 /usr/local/share/cacti/plugins/flowview/flow_collector.php(666): implode('.', false) #1 /usr/local/share/cacti/plugins/flowview/flow_collector.php(356): process_fv9('\x00\t\x00\x13H\xAF\rxfK\x96\x13\x00\n\xCA...', '192.168.222.1:3...') #2 {main}

[TheWitness]

We need a print_r($templates) when template 262 is found. Can you do that?

[TheWitness]

It likely includes a column not previously captured in the case logic.

[TheWitness]

Test again, I've made some changes.

[xmacan]

after fix field 98, I got new warning: Field: 8, Unpack: C4, Length: 4, Pack: ipv4Address Field: 12, Unpack: C4, Length: 4, Pack: ipv4Address Field: 225, Unpack: C4, Length: 4, Pack: ipv4Address Field: 226, Unpack: C4, Length: 4, Pack: ipv4Address Field: 227, Unpack: n, Length: 2, Pack: unsigned16 Field: 228, Unpack: n, Length: 2, Pack: unsigned16 Flow: Processing Record, Remaining: 1 Field: 1, Unpack: J, Length: 8, Pack: unsigned64 PHP Warning: unpack(): Type J: not enough input values, need 8 values but only 1 was provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1059

Warning: unpack(): Type J: not enough input values, need 8 values but only 1 was provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1059 PHP Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1096

Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1096 Field: 23, Unpack: J, Length: 8, Pack: unsigned64 PHP Warning: unpack(): Type J: not enough input values, need 8 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1059

[xmacan]

still problem, but it is maybe problem of my device (fortigate). : Field: 228, Unpack: n, Length: 2, Pack: unsigned16 Flow: Processing Record, Remaining: 0 Flow: Processing v9 Data, Record: 15, FSId:259, FSLen:96 Flow: Data Found, Processing Flow: Template Found: 259 Field: 1, Unpack: J, Length: 8, Pack: unsigned64 Field: 23, Unpack: J, Length: 8, Pack: unsigned64 Field: 2, Unpack: N, Length: 4, Pack: unsigned64 Field: 24, Unpack: N, Length: 4, Pack: unsigned64 Field: 22, Unpack: N, Length: 4, Pack: unsigned32 Field: 21, Unpack: N, Length: 4, Pack: unsigned32 Field: 7, Unpack: n, Length: 2, Pack: unsigned16 Field: 11, Unpack: n, Length: 2, Pack: unsigned16 Field: 10, Unpack: n, Length: 2, Pack: unsigned32 Field: 14, Unpack: n, Length: 2, Pack: unsigned32 Field: 4, Unpack: C, Length: 1, Pack: unsigned8 Field: 98, Unpack: C, Length: 1, Pack: unsigned8 Field: 95, Unpack: C9, Length: 9, Pack: octetArray Field: 66, Unpack: C4, Length: 4, Pack: Field: 65, Unpack: C2, Length: 2, Pack: Field: 89, Unpack: C, Length: 1, Pack: unsigned16 Field: 136, Unpack: C, Length: 1, Pack: unsigned8 Field: 27, Unpack: n8, Length: 16, Pack: ipv6Address Field: 28, Unpack: n8, Length: 16, Pack: ipv6Address Flow: Processing Record, Remaining: 1 Field: 1, Unpack: J, Length: 8, Pack: unsigned64

Warning: unpack(): Type J: not enough input values, need 8 values but only 1 was provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1058

Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1093 Field: 23, Unpack: J, Length: 8, Pack: unsigned64

Warning: unpack(): Type J: not enough input values, need 8 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1058

Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1093 Field: 2, Unpack: N, Length: 4, Pack: unsigned64

Warning: unpack(): Type N: not enough input values, need 4 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1058

Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1093 Field: 24, Unpack: N, Length: 4, Pack: unsigned64

Warning: unpack(): Type N: not enough input values, need 4 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1058

Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1093 Field: 22, Unpack: N, Length: 4, Pack: unsigned32

Warning: unpack(): Type N: not enough input values, need 4 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1058

Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1093 Field: 21, Unpack: N, Length: 4, Pack: unsigned32

Warning: unpack(): Type N: not enough input values, need 4 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1058

Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1093 Field: 7, Unpack: n, Length: 2, Pack: unsigned16

Warning: unpack(): Type n: not enough input values, need 2 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1058

Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1093 Field: 11, Unpack: n, Length: 2, Pack: unsigned16

Warning: unpack(): Type n: not enough input values, need 2 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1058

Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1093 Field: 10, Unpack: n, Length: 2, Pack: unsigned32

Warning: unpack(): Type n: not enough input values, need 2 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1058

Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1093 Field: 14, Unpack: n, Length: 2, Pack: unsigned32

Warning: unpack(): Type n: not enough input values, need 2 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1058

Warning: Trying to access array offset on false in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1093 Field: 4, Unpack: C, Length: 1, Pack: unsigned8

Warning: unpack(): Type C: not enough input values, need 1 values but only 0 were provided in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1058

Fatal error: Uncaught TypeError: implode(): Argument #2 ($array) must be of type ?array, false given in /usr/local/share/cacti/plugins/flowview/flow_collector.php:1081 Stack trace: #0 /usr/local/share/cacti/plugins/flowview/flow_collector.php(1081): implode('', false) #1 /usr/local/share/cacti/plugins/flowview/flow_collector.php(681): process_fv9('\x00\t\x00\x10u\xB0\x85\xF4fW\e\x92\x006{...', '192.168.222.1:3...') #2 {main} thrown in /usr/local/share/cacti/plugins/flowview/flow_collector.php on line 1081

[xmacan]

@TheWitness I couldn't understand my last post. Why is field 1 only unsigned8 instead of unsigned 64? We are not the only ones who have encountered the same problem: https://github.com/Edgio/vflow/issues/66

[TheWitness]

It seems more your firewalls issue now. Maybe I'm missing something, but the firewall provides the template id's, the field id's and the length. We are attempting to use the lengths they provide and we adjust and use the lengths when they differ from the default in the spec.

[TheWitness]

I'm pretty sure this was fixed in the latest commit.

[TheWitness]

Heard from @xmacan that things are back to normal now. So, I'm closing this one.