k6 icon indicating copy to clipboard operation
k6 copied to clipboard

Remote write Prometheus

Open DanielCastronovo opened this issue 1 year ago • 1 comments

Brief summary

whenever I run K6 with Prometheus remote write enabled, I regularly get errors.

k6 version

0.51.0

OS

EKS

Docker version and image (if applicable)

No response

Steps to reproduce the problem

Launch K6 with remote write

Expected behaviour

metric generated

Actual behaviour

level=warning msg="Successful flushed time series to remote write endpoint but it took 15.102561295s whil e flush period is 5s. Some samples may be dropped." nts=40 output="Prometheus remote write"

DanielCastronovo avatar Jul 02 '24 15:07 DanielCastronovo

Any news ? did you want some details ?

DanielCastronovo avatar Jul 12 '24 08:07 DanielCastronovo

Any news ?

DanielCastronovo avatar Nov 29 '24 10:11 DanielCastronovo

Hey @DanielCastronovo 👋🏻

Apologies for coming back to this so late, it completely slid under my radar.

Can you please give us more context as to the conditions in which this happens? Because it's highly dependent on your test script, how you run k6, how and where you run prometheus.

What would be useful would be to see a test script (anonymized), some command-line commands displaying how you run k6, and knowing more about where and how the prometheus instance you have k6 talk to runs is and is configured.

For context this error message essentially indicates that k6 tried to flush samples to Prometheus (it does that periodically), in this case it seems to be trying every 5s, but because the previous flush took more seconds, some samples might have been lost in process. I believe this can be caused by a multitude of factors, hence the need for more context.

Without that it's gonna be hard to get to any conclusions as you can imagine 🙇🏻

oleiade avatar Jan 31 '25 11:01 oleiade