rrdtool-1.x
rrdtool-1.x copied to clipboard
oetiker
RRD database management
I have been looking at InfluxDB - it does time series with consolidation - much like rrd. But it has a SQL like query language which makes it a lot easier to get data out of it as compared with rrd files + perl or whatever and also something called continuous quieries that lets interested parties receive updated data as it arrives for real time graphing and what not.
Having run this on hardware identical to our rrd storage I find the system wanting with regards to performance compared to rrd files and heinous with regards to RAM usage.
If rrdcached was to become rrdd and channel all updates, data for graphing and other queries through it so it could support this kind of thing I think it would be a quite good win. And then you could enter the enterprise-in-a-cloud-web-scale-software community and get rich quick!
We looked at InfluxDB about a year ago with similar results as you describe - then took a misguided detour via ElasticSearch and have now built a solution as you suggest, using rrdtool as the foundation with rrdcached for all data input and output.
You are essentially saying, there should be a REST interface to rrdtool? There is a fundamantal problem with the rrdcached aproach in so much as it does not process the data but rather caches it BEFORE handing it over to rrdtool proper ... so rrdupdatev is not possible ...
what is so important about rrdupdatev?
the problem with influxdb and (afaict) all other tsdbs is that when scaled up they all run into the problem rrdcached solves - write performance. rrdupdatev might even work for rrdcached, if you reduce the number of objects/rrdfiles to which it is applied. eg. when you have 100.000 rrds behind rrdcached that are updated regularly (every 5 minutes) you could have say 2000 of them which do rrdupdatev..
In other words, rrdcached is fundamental for the success of the described use case.
Yes, we have built a simple REST interface infront of rrdcached.
yes I agree write performance is key and rrdcached solves this, but it does not sit at the right position in the processing chain ... data should be processed as it arrives and only the actual writing should be delayed ... in that way the processed data becomes available immediately which makes rrdupdatev work and will also not require data to be read back from disk for short term charts and stuff as it will all sit in the memory cache of rrdcached v2
there is one huge installation I know of, that does not use rrdcached, but rather just alter the linux 'flush to disk' interval up to about 30 minutes ... then linux does all the optimization in core ... they also run a journal with the incoming data and if an instance crashes, they just replay the last 30 minutes of journal.
We have the needed performance with >200K rrd files being written to through rrdcached. Samples go to rrdcached for all 200K files every 5 minutes and then gets written to disk within 30 minutes.
And having just switched storage to SSD disks we have lots of disk performance headroom (we used to use >50% of the HDDs bandwidth, not very comfortable :-)
My influxdb adventure is due to some reporting and signal generation needs and each requires too much perl programming to retrieve data and do some calculations and so on.
RRD is superior with regards to write performance and memory needs, but getting data out of a aggregate of rrd files requires a programmer time every time and this is becoming a drag. Some kind of higher level query interface as well as a way for client to subscribe to data real time through rrdcached (rather than having to force a flush to get hands on data) would go a long way.
the getting the data is at the heart of the design problem with rrdcached I am referring to ...