rrdtool-1.x
rrdtool-1.x copied to clipboard
oetiker
(Random Bug) Uninformative Y scale for "--alt-autoscale" & "--alt-y-grid" graphs
Hello,
Sorry for my english, I use RRDtool 1.4.8 to make a lot of "precision sensors" graphs with autoscale on Y axis. The bug can appear and disappear for few hours according to the variation of data from sensors I added picture from two graph showing and not-showing the bug on Y-scale :
First graph with buggy Y scale :
).
You can see few lower values and precision are missing:(
Same graph without the bug same graph 1 hour in the past :
Near-same data perfectly displayed (one hour in the past) !
Code used for this one :
/usr/bin/rrdtool graph - \
--border=0 \
--grid-dash 1:0 \
--no-minor \
--color BACK#ffffff00 \
--color FONT#000000 \
--color CANVAS#ffffff00 \
--color MGRID#80000040 \
--alt-y-grid \
--imgformat=PNG \
--start='-86400' \
--end='-300' \
--title='Si1000 RF Microcontroller - Robot Battery Voltage' \
--base='1000' \
--height='120' \
--width='500' \
--alt-autoscale \
--vertical-label='volts' \
--slope-mode \
--font TITLE:11: \
--font AXIS:8: \
--font LEGEND:8: \
--font UNIT:8: \
DEF:a='/var/lib/cacti/rra/localhost_batteryvoltage_62.rrd':'batteryvoltage':AVERAGE:start='-86400'-1h \
CDEF:cdefa='a,UN,PREV,a,IF' \
CDEF:cdefe='a,3600,TRENDNAN' \
AREA:cdefa#FF00007F:'Voltage' \
GPRINT:a:LAST:'Current\:%8.2lf %s' \
GPRINT:a:MIN:'Minimum\:%8.2lf %s' \
GPRINT:a:MAX:'Maximum\:%8.2lf %s\n' \
LINE1:cdefe#FF0000FF:''
Second graph with buggy Y scale :
You can see few lower values and precision are missing:(
Same second graph without the bug same graph one hour in the future :
Near-same data perfectly displayed (one hour in the future) !
Code used for second graph :
/usr/bin/rrdtool graph - \
--border=0 \
--grid-dash 1:0 \
--no-minor \
--color BACK#ffffff00 \
--color FONT#000000 \
--color CANVAS#ffffff00 \
--color MGRID#80000040 \
--alt-y-grid \
--imgformat=PNG \
--start='-86400' \
--end='-300' \
--title='Localhost - Audio RMS Amplitude' \
--base='1000' \
--height='120' \
--width='500' \
--alt-autoscale \
--vertical-label='percent' \
--slope-mode \
--font TITLE:11: \
--font AXIS:8: \
--font LEGEND:8: \
--font UNIT:8: \
DEF:a='/var/lib/cacti/rra/localhost_audio_59.rrd':'audio':AVERAGE:start='-86400'-1h \
CDEF:cdefa='a,UN,PREV,a,100,*,IF' \
CDEF:cdefb='a,100,*' \
CDEF:cdefe='a,3600,TRENDNAN,100,*' \
AREA:cdefa#8D00BA7F:'Audio' \
GPRINT:cdefb:LAST:'Current\:%8.2lf %s' \
GPRINT:cdefb:MIN:'Minimum\:%8.2lf %s' \
GPRINT:cdefb:MAX:'Maximum\:%8.2lf %s\n' \
LINE1:cdefe#942D0CFF:''
Another interesting case :
There is a big hole on my Y-scale:(
Live data on my website, it show the bug multiple time by day on few graph : https://www.serveurperso.com/?page=cacti You can move in the past or future with click on any graph border. The bug can appear and disappear.
Please help to debug this annoying Y scale generator bug on RRDtool 1.x
Pascal
you are in luck ... rrdtool 1.4.8 is about 100000 years old ... treat yourself to version 1.6. Nice charts btw!
If I open this case this is because the problem is still on the 1.6 sorry for missing information
I compiled my RRDtool from this repo and checkouted inside origin/1.6 The v1.4.8 is because this is the official debian jessie version installed with Cacti (mistake rrdtool --version).
this is odd, because a bunch of these problems was fixed in the 1.5 series ... please provide a sample script to reproduce your problem ... ideally an xml dump of your rrd file and the graph call to create the chart ...
the smaller and simpler the input the easier to fix
sorry I opened another issue please remove duplicate No problem to provide data I do this tonight
I can start from this URL : https://www.serveurperso.com/cacti/graph_image.php?local_graph_id=50&rra_id=5&graph_width=527&graph_height=223&graph_nolegend=true&graph_start=1478077809&graph_end=1478099409&fond=e0e0ff
and make sure you are actually running 1.6 ... we had similar reports and eventually the user found that he was not running 1.6 ... often problem related to old versions of the librrd hanging round
I 100% sure because I renamed my binary from debian and defined the full path of new one If I rename the new one PHP say "the image can't displayed because contain errors" in french:) For sure I run the 1.6
I created the robot battery voltage RRA few days ago and there is the bug between these timestamp graph_start=1478077809 graph_end=1478099409
I make a full XML from my RRA, and try to eliminate a maximum of useless data as possible
localhost_batteryvoltage_62.rrd localhost_batteryvoltage_62.rrd.xml
They are untouched files for my setup, this graph is recent without a lot of data. Do you want other action (please give me the method)
RRDTool Command:
/root/git/rrdtool-1.x/src/rrdtool graph - \
--border=0 \
--grid-dash 1:0 \
--no-minor \
--color BACK#ffffff00 \
--color FONT#000000 \
--color CANVAS#ffffff00 \
--color MGRID#80000040 \
--alt-y-grid \
--imgformat=PNG \
--start='-86400' \
--end='-300' \
--title='Si1000 RF Microcontroller - Robot Battery Voltage' \
--base='1000' \
--height='120' \
--width='500' \
--alt-autoscale \
--vertical-label='volts' \
--slope-mode \
--font TITLE:11: \
--font AXIS:8: \
--font LEGEND:8: \
--font UNIT:8: \
DEF:a='/var/lib/cacti/rra/localhost_batteryvoltage_62.rrd':'batteryvoltage':AVERAGE:start='-86400'-1h \
CDEF:cdefa='a,UN,PREV,a,IF' \
CDEF:cdefe='a,3600,TRENDNAN' \
AREA:cdefa#FF00007F:'Voltage' \
GPRINT:a:LAST:'Current\:%8.2lf %s' \
GPRINT:a:MIN:'Minimum\:%8.2lf %s' \
GPRINT:a:MAX:'Maximum\:%8.2lf %s\n' \
LINE1:cdefe#FF0000FF:''
RRDTool Says:
OK
does it need all these options ?
also an absolute start and end time would be helpful
Cacti time : graph_start=1478077809 graph_end=1478099409
If you want I can dump exact command line for the time of this bug + minify it, but tonight, I anticipated giving you what I have
This is the exact cmdline used to create this graph (timestamp are the same) I just patched my rrd.php from cacti to write a file from $command_line var.
graph -
--border=0
--grid-dash 1:0
--no-minor
--color BACK#e0e0ff00
--color FONT#000000
--color CANVAS#e0e0ff00
--color MGRID#80000040
--alt-y-grid
--imgformat=PNG
--start='1478077809'
--end='1478099409'
--title='Si1000 RF Microcontroller - Robot Battery Voltage'
--base='1000'
--height='223'
--width='527'
--alt-autoscale
--no-legend COMMENT:"From 2016/11/02 10\:10\:09 To 2016/11/02 16\:10\:09\c" COMMENT:" \n"
--vertical-label='volts'
--slope-mode
--font TITLE:7.7:
--font AXIS:8:
--font LEGEND:8:
--font UNIT:8: DEF:a='/var/lib/cacti/rra/localhost_batteryvoltage_62.rrd':'batteryvoltage':AVERAGE:start='1478077809'-1h CDEF:cdefa='a,UN,PREV,a,IF' CDEF:cdefe='a,3600,TRENDNAN' AREA:cdefa#FF00007F:'' LINE1:cdefe#FF0000FF:''
This bug is easy to reproduce anywhere there is :
1) --alt-autoscale combined with --alt-y-grid parameter 2) small variance of data 3) very small variance of data (floating/fixed point values) is buggy but not mandatory : bug is also on integer scales 4) I encounter only for no unit prefixed values (I can't find bug for 1 k / 1 M values)
Edit : I tested "--alt-autoscale" without "--alt-y-grid" and I get this already know problem :
-> This is why I patched my Cacti rrd.php with "--alt-y-grid".
The rrdtool adaptive scale with "--alt-autoscale" and "--alt-y-grid" is the bug, other "cosmetic" parameter seems to have no impact (http://www.serveurperso.com/cacti/graph_image.php?local_graph_id=50&rra_id=5&graph_width=1556&graph_height=682&graph_nolegend=true&graph_start=1478078713&graph_end=1478100313&fond=ffffff any resolution, color, with or without RPN math...)
I think this is a complex routine to make an always readable adaptive scale:(
I also had this problem with the scaling. In my case, I am graphing temperature over a time period. When the data in the given timeframe is in the range 20 to 25, the scale is not generated correctly:
But once the data goes below 20, and the graph scale expands to show 10 - 25, the scale is correct:
The graphs are generated the same way, with only the end times differing. eg.
rrdtool graph monday.png \ --imgformat PNG \ --start 1477832400 \ --end 1477918800 \ -w 800 -h 400 \ DEF:temp=ds18b20.rrd:temp:AVERAGE \ LINE1:temp#000000
I am using version 1.6.0
Yes it fixes this known problem but it make another bug : why I opened this case. This case is about the --alt-autoscale combined with --alt-y-grid bug (I edited my post before the your one to clarify.)
Next days look your graph frequently or move around it, for sure you can get the bug of this github issue.
For computer / IT supervision this bug is near never reproduced ! -> But for any Scientific use / robotics / weather / precision sensors / small values with small variance, it is sufficiently frequent and annoying to make RRDtool unusable :(
I think this is a high priority bug, please help -> Can I use RRDtool 2.x with CACTI (about cmdline compatibility) ?
@Serveurperso If I run your example with the latest rrdtool code from github:
$ curl https://www.serveurperso.com/temp/localhost_batteryvoltage_62.rrd.xml \
|./rrdtool restore - battery.rrd
$ ./rrdtool graph battery.png --border=0 --grid-dash 1:0 --no-minor --color BACK#ffffff00 \
--color FONT#000000 --color CANVAS#ffffff00 --color MGRID#80000040 \
--alt-y-grid --imgformat=PNG --start=1478077809 --end=1478099409 \
--title='Si1000 RF Microcontroller - Robot Battery Voltage' --base='1000' \
--height='120' --width='500' --alt-autoscale --vertical-label='volts' --slope-mode \
--font TITLE:11: --font AXIS:8: --font LEGEND:8: --font UNIT:8: \
DEF:a='battery.rrd':'batteryvoltage':AVERAGE:start='-86400'-1h \
CDEF:cdefa='a,UN,PREV,a,IF' CDEF:cdefe='a,3600,TRENDNAN' \
AREA:cdefa#FF00007F:'Voltage' \
GPRINT:a:LAST:'Current\:%8.2lf %s' \
GPRINT:a:MIN:'Minimum\:%8.2lf %s' \
GPRINT:a:MAX:'Maximum\:%8.2lf %s\n' \
LINE1:cdefe#FF0000FF:''
with --alt-y-grid and --alt-autoscale the result looks like this:
with only --alt-y-grid the result looks like this:
with only --alt-autoscale the result looks like this:
and without any special grid or scaling the result looks like this:
so the only thing one might say is that --alt-autoscale does not show enough precision in the y axis but this can be remedied by using --alt-y-grid OR --left-axis-format="%.1lf" (as shown in the example below)
I can't build the last master :
(root|~/git/rrdtool-1.x) make
Making all in po
make[1]: Entering directory '/root/git/rrdtool-1.x/po'
make[1]: Leaving directory '/root/git/rrdtool-1.x/po'
Making all in src
make[1]: Entering directory '/root/git/rrdtool-1.x/src'
make all-am
make[2]: Entering directory '/root/git/rrdtool-1.x/src'
CC librrd_la-rrd_version.lo
CC librrd_la-rrd_last.lo
CC librrd_la-rrd_lastupdate.lo
CC librrd_la-rrd_first.lo
CC librrd_la-rrd_dump.lo
rrd_dump.c:465:5: warning: no previous prototype for 'rrd_dump_opt_r' [-Wmissing-prototypes]
int rrd_dump_opt_r(
^
CC librrd_la-rrd_flushcached.lo
CC librrd_la-rrd_fetch.lo
CC librrd_la-rrd_fetch_cb.lo
CC librrd_la-rrd_resize.lo
CC librrd_la-rrd_tune.lo
CC librrd_la-rrd_graph.lo
CC librrd_la-rrd_graph_helper.lo
CC librrd_la-rrd_xport.lo
CC librrd_la-rrd_gfx.lo
CC librrd_la-pngsize.lo
CC librrd_la-rrd_restore.lo
CC librrdupd_la-optparse.lo
CC librrdupd_la-rrd_strtod.lo
CC librrdupd_la-rrd_create.lo
CC librrdupd_la-rrd_parsetime.lo
CC librrdupd_la-rrd_hw.lo
CC librrdupd_la-rrd_hw_math.lo
CC librrdupd_la-rrd_hw_update.lo
CC librrdupd_la-rrd_diff.lo
CC librrdupd_la-rrd_format.lo
CC librrdupd_la-rrd_info.lo
CC librrdupd_la-rrd_error.lo
CC librrdupd_la-rrd_open.lo
CC librrdupd_la-rrd_client.lo
CC librrdupd_la-rrd_nan_inf.lo
CC librrdupd_la-rrd_rpncalc.lo
CC librrdupd_la-rrd_utils.lo
CC librrdupd_la-rrd_snprintf.lo
CC librrdupd_la-rrd_update.lo
CC librrdupd_la-rrd_modify.lo
CC librrdupd_la-rrd_thread_safe.lo
CCLD librrdupd.la
libtool: link: warning: `-version-info/-version-number' is ignored for convenience libraries
CCLD librrd.la
CC rrd_tool.o
CCLD rrdtool
rrd_tool.o: In function `HandleInputLine':
/root/git/rrdtool-1.x/src/rrd_tool.c:696: undefined reference to `rrd_list'
collect2: error: ld returned 1 exit status
Makefile:714: recipe for target 'rrdtool' failed
make[2]: *** [rrdtool] Error 1
make[2]: Leaving directory '/root/git/rrdtool-1.x/src'
Makefile:552: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/root/git/rrdtool-1.x/src'
Makefile:557: recipe for target 'all-recursive' failed
make: *** [all-recursive] Error 1
(root|~/git/rrdtool-1.x)
make OK from a fresh clone / configure; sorry
But bug not solved on master 1.5.999
RRDtool 1.5.999 Copyright by Tobias Oetiker [email protected] Compiled Nov 3 2016 14:24:46
Why not 1.6.x ?
What are you trying to tell me ? In my previous post I showed you the output you get from running your code with the current master of rrdtool-1.x ... if you use that version you are quite likely ging to get the same results ...
ps. thanks for the 1.5.999 hint ... fixed
The bug is still here for me : the problem is another combination or graph resolution ? very strange I already tested this bug with any resolution. Can you test with my cmdline (remove stdout) ? Or can you test your cmdline without legend ?
rrdtool graph battery.png \
--border=0 \
--grid-dash 1:0 \
--no-minor \
--color BACK#e0e0ff00 \
--color FONT#000000 \
--color CANVAS#e0e0ff00 \
--color MGRID#80000040 \
--alt-y-grid \
--imgformat=PNG \
--start='1478077809' \
--end='1478099409' \
--title='Si1000 RF Microcontroller - Robot Battery Voltage' \
--base='1000' \
--height='223' \
--width='527' \
--alt-autoscale \
--no-legend COMMENT:"From 2016/11/02 10\:10\:09 To 2016/11/02 16\:10\:09\c" COMMENT:" \n" \
--vertical-label='volts' \
--slope-mode \
--font TITLE:7.7: \
--font AXIS:8: \
--font LEGEND:8: \
--font UNIT:8: \
DEF:a='battery.rrd':'batteryvoltage':AVERAGE:start='1478077809'-1h \
CDEF:cdefa='a,UN,PREV,a,IF' CDEF:cdefe='a,3600,TRENDNAN' AREA:cdefa#FF00007F:'' \
LINE1:cdefe#FF0000FF:''
Tested from your fresh :
RRDtool 1.6.999 Copyright by Tobias Oetiker [email protected] Compiled Nov 3 2016 15:26:28
Hi, Any news ? can you give me the rrdgraph function name that make this particular dynamic scale ?
sorry ... too much work for $$$ to dig into this more deeply ... until now
look for calc_horizontal_grid in rrd_graph.c the code is in im->extra_flags & ALTYGRID
the problem is that it is contributed code, its structure is quite alien to me ... I suspect that the best would be to rewrite it from scratch ... reading it I get the impression that it is more complex than it needs to be.
I get a similar issue with 1.7.2. The values are around 10 G (slightly less up to slightly greater). If I don't use special options for the y-axis, the generated Y range in the graph is 9 G to 20 G, which is not satisfactory because the small variations are not visible. The --alt-y-grid option gives the same graph, except that the Y step for the labels is now 2 G instead of 1 G. The --alt-autoscale option yields a graph with an acceptable scale, but only "10 G" is shown, thus one cannot know the other values, i.e. the actual scale; the y-axis should show at least 2 values, and with enough decimals to be able to distinguish them (see #1081 for an example with 2 identical labels).
To reproduce: test.rrd.gz with the uncompressed file and a command like (e.g. with --alt-autoscale):
rrdtool graph out.png --alt-autoscale --start 20220101 --end 20220103 -a PNG -h 200 'DEF:kavg=test.rrd:data:AVERAGE' 'CDEF:avg=kavg,1024,*' 'DEF:kmax=test.rrd:data:MAX' 'CDEF:max=kmax,1024,*' 'AREA:avg#00ff00' 'LINE1:max#ff0000'
And the obtained graphs in the 3 cases described above:
