json_exporter icon indicating copy to clipboard operation
json_exporter copied to clipboard

how to scrape two json files that have different metrics

Open ss666 opened this issue 3 years ago • 13 comments

how to use two configuration files to convert/scrape two different json files in one program, or I have to start two programs with using two different port numbers

ss666 avatar Jul 08 '21 10:07 ss666

Also interested in this.

ArjonBu avatar Jul 22 '21 14:07 ArjonBu

this is currently possible as long as the json paths are unique between the two endpoints, but it creates a mess in the error logs, due to all the error messages about not being able to find paths / extract values...

Would be great to be able to tag metrics w/ endpoint URLs (or URL regexes) to which they will be applied.

rdugan avatar Jul 25 '21 09:07 rdugan

Any updates on this feature ?

T1MOXA avatar Sep 24 '21 15:09 T1MOXA

me too, Also interested in this.

shulianwang avatar Oct 15 '21 15:10 shulianwang

I'd also like this and the only way I can think to do it (in kubernetes) is running multiple instances of json_exporter with multiple configmaps. Unless anyone has a better idea.

nicklinnell avatar Nov 29 '21 14:11 nicklinnell

This is a duplicate of https://github.com/prometheus-community/json_exporter/issues/14, which was closed for some reason. Commenting to subscribe!

EvilJordan avatar Jan 08 '22 01:01 EvilJordan

+1 on this. I need to extract values from multiple json endpoints with conflicting paths.

pinpox avatar Jan 21 '22 11:01 pinpox

Commented on #14 already, but chiming in here, I also greatly support this enhancement. I am now using 6 Dockerized instances of this exporter within my monitoring stack in order to poll multiple endpoints with differing authentication, as there is no way to distinguish between sending auth to any particular endpoint. This is not very scalable

crockk avatar Mar 17 '22 18:03 crockk

Given that there's no movement on this, and I have no capabilities of writing something myself, are there any recommended alternatives for polling multiple endpoints?

EvilJordan avatar Aug 26 '22 01:08 EvilJordan

I believe I have found a solution, albeit a bit labor intensive (due to the custom processing required): Instead of relying on json_exporter at all, configure node_exporter to scrape a local directory for *.prom files with its textfile-collector: https://github.com/prometheus/node_exporter#textfile-collector

A recurring cron would hit every desired endpoint, parse, and output the data to a file (or multiple files!) in the appropriate format for node_exporter to scrape.

Maybe not as elegant as json_exporter is attempting to be, but this should solve the problem. I'll let you know how it goes!

EvilJordan avatar Aug 26 '22 03:08 EvilJordan

Follow up: This totally worked. No more awful errors. I wrote a simple shell script that curls the necessary endpoints, pipes the data through jq to parse, and then writes out to a file. Then, I have a cron that runs every minute to execute the script and replace the file with the latest data. Dead-simple and now my logs are ::chefs kiss::

EvilJordan avatar Sep 17 '22 02:09 EvilJordan

This should now be possible with the added support for modules in https://github.com/prometheus-community/json_exporter/pull/146

rustycl0ck avatar Nov 30 '22 11:11 rustycl0ck

Brilliant! Thanks for this!

rdugan avatar Feb 23 '23 00:02 rdugan