95th Percentile Caculation for Aggregate (Last Month) is not accurate.

[iskandarbasman]

Describe the bug

95th Percentile Caculation for Aggregate (Last Month) is not accurate.

To Reproduce

Steps to reproduce the behavior: Cacti Version 1.2.25

Created a test graph on by test cacti.

1. Using two existing 95th Percentile graphs.
2. Created a Aggregate 95th graph.
3. Used Preset: Last Month
4. See cacti 95th percentile calculated as 711.16 Mbits

Download the CSV export of the same graph data. Noted that Cacti 95th value as in the graph. "Nth Percentile 711157443 |95:bits:0:aggregate_peak:2|"

Compared using the same data to calculate 95th Percentile. Excel formula is getting : 502,123,554.75 Manual calculation using approximately 95% of total sample: 508305728.43

95th percentile by Excel and Manual calculation is close to each other. 95th percentile by Cacti is over by 200Mbits.

Attached the excel file:

Aggregate - test 19 mar 2024.xlsx

` RRDtool Command:

/usr/bin/rrdtool graph -
--imgformat=PNG
--start='1708307700'
--end='1710813147'
--pango-markup
--title='Aggregate - test 19 mar 2024'
--vertical-label='bits per second'
--slope-mode
--base=1000
--height=200
--width=700
--rigid
--alt-autoscale-max
--lower-limit='0'
COMMENT:"From 2024-02-19 09:55:00 To 2024-03-19 09:52:27\c"
COMMENT:" \n"
--border 1
--slope-mode
--watermark 'Generated by SPTEL Cacti®'
DEF:a='/var/www/html/cacti/rra/1nete-ne20e-s1_traffic_in_2562.rrd':'traffic_in':LAST
DEF:b='/var/www/html/cacti/rra/1nete-ne20e-s1_traffic_in_2570.rrd':'traffic_in':LAST
DEF:c='/var/www/html/cacti/rra/1nete-ne20e-s1_traffic_in_2562.rrd':'traffic_out':LAST
DEF:d='/var/www/html/cacti/rra/1nete-ne20e-s1_traffic_in_2570.rrd':'traffic_out':LAST
CDEF:cdefa='a,8,'
CDEF:cdefe='b,8,'
CDEF:cdefi='c,8,'
CDEF:cdefbc='d,8,'
CDEF:cdefbj='TIME,1710806271,GT,a,a,UN,0,a,IF,IF,TIME,1710806271,GT,b,b,UN,0,b,IF,IF,+,8,'
CDEF:cdefce='TIME,1710806271,GT,c,c,UN,0,c,IF,IF,TIME,1710806271,GT,d,d,UN,0,d,IF,IF,+,8,'
AREA:cdefa#00CF007F:'1NETE-NE20E-S1'
GPRINT:cdefa:LAST:'Current:%8.2lf %s'
GPRINT:cdefa:AVERAGE:'Average:%8.2lf %s'
GPRINT:cdefa:MAX:'Maximum:%8.2lf %s\n'
AREA:cdefe#00CF007F:'1NETE-NE20E-S1':STACK
GPRINT:cdefe:LAST:'Current:%8.2lf %s'
GPRINT:cdefe:AVERAGE:'Average:%8.2lf %s'
GPRINT:cdefe:MAX:'Maximum:%8.2lf %s\n'
AREA:cdefi#002A97FF:'1NETE-NE20E-S1'
GPRINT:cdefi:LAST:'Current:%8.2lf %s'
GPRINT:cdefi:AVERAGE:'Average:%8.2lf %s'
GPRINT:cdefi:MAX:'Maximum:%8.2lf %s\n'
AREA:cdefbc#002A97FF:'1NETE-NE20E-S1':STACK
GPRINT:cdefbc:LAST:'Current:%8.2lf %s'
GPRINT:cdefbc:AVERAGE:'Average:%8.2lf %s'
GPRINT:cdefbc:MAX:'Maximum:%8.2lf %s\n'
COMMENT:' \n'
COMMENT:' \n'
COMMENT:' \n'
LINE1:cdefbj#00CF00FF:
LINE1:cdefbj#00CF007F:'Total '
GPRINT:cdefbj:LAST:'Current:%8.2lf %s'
GPRINT:cdefbj:AVERAGE:'Average:%8.2lf %s'
GPRINT:cdefbj:MAX:'Maximum:%8.2lf %s\n'
LINE1:cdefce#002A97FF:
LINE1:cdefce#002A977F:'Total '
GPRINT:cdefce:LAST:'Current:%8.2lf %s'
GPRINT:cdefce:AVERAGE:'Average:%8.2lf %s'
GPRINT:cdefce:MAX:'Maximum:%8.2lf %s\n'
COMMENT:' \n'
HRULE:711157442.95#FF0000FF:'95th Percentile'
COMMENT:'(711.16 mbit in+out)'

RRDtool Command lengths = 2253 characters. RRDtool Says: OK `

[aliuzzz]

1.2.26 is ok

[iskandarbasman]

I saw there was a major bug fix aggregate graphs and percentiles for 1.2.26. So far trying it out has been positive (95th percentile value is more accurate) but its seems i have to recreate the aggregate graphs for it to use this new N-th percentile template ?

Old Graph

New Graph

[TheWitness]

We will table this discussion for 1.2.27. We will restart the conversation after release.

[DFlexy]

Dear colleague, I also identified the problem in version 27 and changed the file below to correct it, I hope it helps you. Make a backup first ;) and then let me know if it worked.

https://github.com/Cacti/cacti/commit/d7b586c105ccf0d8acad7488bbde7145dc176fd3?diff=split&w=0

[TheWitness]

@DFlexy, do you mean you backed off the changes and reverted to the previous. Please be more specific.

[DFlexy]

@DFlexy, você quer dizer que recuou nas alterações e voltou para o anterior. Por favor seja mais específico.

Yes, I pulled a file that I had helped the developer correct the percentiles in aggregates. With that, my aggregate graphs started working again.

[TheWitness]

Would you mind please doing a pull request on this one? Do it against 1.2.x. Thanks!!

[iskandarbasman]

Would you mind please doing a pull request on this one? Do it against 1.2.x. Thanks!!

Is there a fix for 1.2.27 ? Getting an error save aggregated after updating api_aggregate.php manually in 1.2.27.

I tried to download and install cacti-develop branch but hitting into other issues.

[TheWitness]

Run the audit_database.php cli script starting with the --report option and finishing with the --repair option. Post your results.

[iskandarbasman]

Run the audit_database.php cli script starting with the --report option and finishing with the --repair option. Post your results.

You mean run this on cacti 1.2.27 with the patched api_aggregate.php ?

[TheWitness]

Yes, did you open a pull request yet?

[iskandarbasman]

Yes, did you open a pull request yet?

Ehh... I think you meant the pull request from @DFlexy I am just a tester not a debugger lol.

[TheWitness]

Well opening one is real simple. One of you should do it.

[TheWitness]

Dear colleague, I also identified the problem in version 27 and changed the file below to correct it, I hope it helps you. Make a backup first ;) and then let me know if it worked.

d7b586c?diff=split&w=0

@DFlexy,

Can you do a pull request for me as I don't know if you rolled back that patch or something else.

Larry

[TheWitness]

The only thing material in that patch was adding back the static declaration.

function aggregate_handle_ptile_type($member_graphs, $skipped_items, $local_graph_id, $_total, $_total_type) {
    static $special_comments = null;
    static $special_hrules   = null;

[iskandarbasman]

Well opening one is real simple. One of you should do it.

I am not sure if I am looking at the code correctly. I think was looking at the wrong branch for fix #5555 before.

In 1.2.X https://github.com/Cacti/cacti/commit/398e58d0588826424d304cd4eaaf47182251c787?diff=unified&w=0 Seems like all the changes mentioned by @DFlexy is already implemented in 1.2.27 code.

[DFlexy]

Good morning, I am currently using version 1.2.27 and since I had the problem mentioned, I inserted the files manually and after that everything went back to normal. So at the moment I am using the file mentioned in patch #5555 as mentioned in the link. https://github.com/Cacti/cacti/issues/5555

[TheWitness]

@DFlexy, Can you upload your version of the file?

[DFlexy]

@DFlexy, Can you upload your version of the file?

Here

Rename api_aggregate.php.txt for api_aggregate.php api_aggregate.php.txt

Here is a print of a graph taken today

[aliuzzz]

It's actually a problem with "aggregate_peak", just replace it with "aggregate"

[iskandarbasman]

It's actually a problem with "aggregate_peak", just replace it with "aggregate"

how are you doing that ?

[TheWitness]

Okay, that file you uploaded was for develop. So, I think this is fixed for both of you guys.