reporting to time series db like eg influx / prometheus

[f1-outsourcing]

I am new to gstreamer, and came across your video about profiling[1] because I was looking for some stats reporting on pipelines. So maybe my question is not applicable. But did you ever think about using time series databases and dashboards in eg grafana?

[1] https://www.youtube.com/watch?v=6HGDkMuqRvs

[michaelgruner]

Oh I would love a smarter data visualization dashboard. I wasn't familiar with Grafana and indeed looks looks very interesting. It certainly applies, may I interest you in starting this effort? :)

[f1-outsourcing]

Hmm, I am not really sure if my help would be sufficient since I am not really familiar with gst-shark, I did not even run stats yet. Grafana draws graphs from data stored in supported backends like prometheus, influx but also things like mysql. Since you are having time series data, it is probably interesting to choose a storage backend like influx or prometheus.

Prometheus is in it's default setup is scrapping data. Which means prometheus would connect to gst-shark and retrieves the data. I do not think you would want to start with this.

The default of influx is that it is receiving data from the clients and stores it. I think this is the easiest way to start. I have some shell script that stores data in influx with this

curl -i -XPOST "http://$SERVER:$PORT/write?db=$DATABASE" --data-binary ''"$MEASUREMENT"',host='"$LOGHOSTNAME"' value="'"$LOGMESSAGE"'" '"$DATEEPO"''

As I do not know what gst-shark is outputting and how much, one can think of buffering some data and send it as one request, because 'real time' graphs are not really interesting (I assume, grafana has a min. refresh rate of 5s).

If you want to experiment with an influx and grafana environment and do not want to setup a development environment, I can open some firewall ports of our environment.