promxy icon indicating copy to clipboard operation
promxy copied to clipboard

Supporting VictoriaMetric's Extended PromQL

Open AeroNotix opened this issue 5 years ago • 34 comments

There are some nice convenience functions here: https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/ExtendedPromQL which make writing queries, alerts etc much simpler.

When I try to use them with promxy, I get errors such as:

Error executing query: invalid parameter 'query': parse error at char 3: unknown function with name "ru" for a query such as: ru(kubelet_volume_stats_available_bytes, kubelet_volume_stats_capacity_bytes).

I understand you are using the underlying prometheus libraries for a lot of the query layer and it's likely that layer which is bubbling this error - but would promxy consider adding support for VM's extended PromQL?

AeroNotix avatar Dec 04 '19 11:12 AeroNotix

I'd definitely consider it, I actually have an issue upstream (https://github.com/VictoriaMetrics/VictoriaMetrics/issues/56) to move the extended promql stuff into a more re-usable library. At this point I don't have the driving use case to spend my time rearchitecting that library to work with promql's types, but I'm definitely for it being in there eventually, PRs are definitely welcome!

jacksontj avatar Dec 05 '19 18:12 jacksontj

Cool, worthwhile changing the labels to feature or similar?

AeroNotix avatar Dec 05 '19 18:12 AeroNotix

Done :)

jacksontj avatar Dec 06 '19 04:12 jacksontj

FYI, we already have a PR that moves extended PromQL code into a separate library - https://github.com/VictoriaMetrics/VictoriaMetrics/pull/256

valyala avatar Dec 09 '19 11:12 valyala

FYI, PromQL parser from VictoriaMetrics has been extracted into a standalone library - see docs.

valyala avatar Dec 25 '19 20:12 valyala

From a quick look it'll require some more work to be supported in promxy, as promxy actually uses a forked version of promql with support for NodeReplacer (a mechanism to rewrite the query tree post-parse pre-execution). I have patches to add that functionality on prometheus that I maintain, presumably we could add something similar here.

jacksontj avatar Dec 26 '19 16:12 jacksontj

This sounds good!

valyala avatar Dec 26 '19 17:12 valyala

FYI, the MetricsQL parser has been moved to a separate repository with its own release schedule - https://github.com/VictoriaMetrics/metricsql . This should significantly simplify its' integration into third-party projects such as Promxy.

valyala avatar Jun 20 '20 17:06 valyala

I'd be very interested in this.

@jacksontj if you can point me at the right bits of code, I'm sure i could come up with a PR.

My company would be happy to sponsor this issue if you'd prefer that.

jeromegn avatar Jul 15 '20 11:07 jeromegn

The place that I think we'd want to implement this is to implement the Engine interface (https://github.com/jacksontj/promxy/blob/master/cmd/promxy/main.go#L188) and then add an option to swap between those.

TBH I haven't spent a lot of time looking into it so I'm not sure exactly how difficult it'll be. I'd be more than happy to review a PR and if you are interested in other options we can discuss those as well :)

jacksontj avatar Jul 22 '20 17:07 jacksontj

I use Promxy in conjunction with VictoriaMetrics. I really lack the support of MetricsQL in Promxy. I'll be glad to see this in the next release. Hope it'll be implemented soon! )

sdryga avatar Aug 05 '20 17:08 sdryga

@jacksontj I use Promxy in conjunction with VictoriaMetrics too. I really lack the support of MetricsQL in Promxy. I'll be glad to see this in the next release. Hope it'll be implemented soon! )

davidsome avatar Dec 21 '20 07:12 davidsome

+1 for supporting of MetricsQL in Promxy

dnull88 avatar Mar 24 '21 11:03 dnull88

+1 for supporting of MetricsQL in promxy

IrekFasikhov avatar May 31 '21 09:05 IrekFasikhov

@jacksontj Hi, Thomas. Need some help for the implementation?

IrekFasikhov avatar May 31 '21 09:05 IrekFasikhov

+1 very demanded functionality

ihard avatar Jul 21 '21 15:07 ihard

+1 Would love to see this implemented. Only thing stopping me from switching to promxy. 🙂

jc16180 avatar Jul 27 '21 14:07 jc16180

+1 for supporting of MetricsQL in promxy

anvpetrov avatar Jul 29 '21 15:07 anvpetrov

+1 for supporting of MetricsQL in promxy

autokilla47 avatar Aug 03 '21 10:08 autokilla47

+1

valdis932 avatar Aug 10 '21 20:08 valdis932

yikes, lads, +1 city in here.

Pull out the editor and get to work. Sheesh.

AeroNotix avatar Aug 12 '21 01:08 AeroNotix

https://github.com/jacksontj/promxy/issues/253#issuecomment-569083961 @jacksontj where are these patches?

AeroNotix avatar Aug 12 '21 01:08 AeroNotix

+1 for supporting of MetricsQL in promxy

G-Asura avatar Aug 17 '21 02:08 G-Asura

+1 for supporting of MetricsQL in promxy

sc7565 avatar Aug 25 '21 18:08 sc7565

+1 for supporting MetricsQL in promxy. This is extremely important for us to be able to use promxy in our production environment that has multiple victoriametrics clusters. We have 1000s of dashboards built using promQL and metricsQL and can not change them.

tiagosanto737 avatar Dec 14 '21 13:12 tiagosanto737

Does anyone work on this? If no, I'd like to take it)

hagen1778 avatar Dec 17 '21 08:12 hagen1778

Does anyone work on this? If no, I'd like to take it)

@valyala were you able to make any progress? :)

V1pr avatar Jan 04 '23 14:01 V1pr

Related, but not the same. With the 0.79.0 release we are getting the following error when querying absent_over_time against a Victoria Metrics backend.

expanding series: remote server http://vmselect.victoriametrics:8481/select/1/prometheus/api/v1/read returned HTTP status 400 Bad Request: remoteAddr: "10.24.11.33:51258"; requestURI: /select/1/prometheus/api/v1/read; unsupported path requested: "/select/1/prometheus/api/v1/read"

I understand VM doesn't support remote read. Any workaround?

psalaberria002 avatar Apr 13 '23 13:04 psalaberria002

I understand VM doesn't support remote read. Any workaround?

set remote_read: false in promxy config ;)

oOHenry avatar Apr 13 '23 13:04 oOHenry

Hey, Is it still in progress? @valyala @jacksontj

mohammadkhavari avatar Apr 29 '23 11:04 mohammadkhavari