jmeter-prometheus-plugin icon indicating copy to clipboard operation
jmeter-prometheus-plugin copied to clipboard

How to increase data retention duration in Prometheus?

Open Sravanthi0614 opened this issue 2 years ago • 10 comments

HI,

I am using Prometheus-2.32.1 version, in the prometheus.yml file how can I increase the duration for data retention.

Thanks, Sravanthi MS

Sravanthi0614 avatar Feb 14 '23 13:02 Sravanthi0614

It appears to be this flag --storage.tsdb.retention.time.

https://prometheus.io/docs/prometheus/latest/storage/

johrstrom avatar Feb 14 '23 14:02 johrstrom

Using this tag in prometheus.yml file, it is throwing error. Kindly help me where and how I need to configure this tag.

Sravanthi0614 avatar Feb 14 '23 15:02 Sravanthi0614

I think it's a flag that you pass to Prometheus when you boot it up.

johrstrom avatar Feb 14 '23 15:02 johrstrom

Can you please brief me on where the parameters are passed to Prometheus(eg:setup or bin or etc folders or any file - as i couldn't find them) during the boot up.

Sravanthi0614 avatar Feb 15 '23 13:02 Sravanthi0614

How does your Prometheus server start? Do you have a systemd unit file?

johrstrom avatar Feb 15 '23 14:02 johrstrom

I am starting Prometheus as a service, by clicking on prometheus.exe there is no systemd unit file

Sravanthi0614 avatar Feb 16 '23 14:02 Sravanthi0614

FYI, i am running on windows machine.

Sravanthi0614 avatar Feb 16 '23 14:02 Sravanthi0614

Got it. So I'm unfamiliar with how Windows made that icon that you click. But if it's a text file that you see prometheus.exe in, that's where you'd edit.

Basically you want to issue this to use 2 weeks, instead of just issuing prometheus.exe.

prometheus.exe --storage.tsdb.retention.time=2w

johrstrom avatar Feb 16 '23 14:02 johrstrom

I successfully started prometheus with the following argument in the windows command prompt. prometheus.exe --storage.tsdb.retention.time=1y

and the console no longer announces prometheus is using the default 15 days data retention. But it is not confirming its using 1 year data retention in the console (it doesn't report anything about data retention, this time), and I also saw in the console multiple ..."Deleting obsolete block"... lines. We dont have data that is older than 1 year but we do have data that goes back to 15 days old threshold, so it looks like it might still be behaving as if the data retention is still set for 15 days. To at least help figure out what is going on here, is there a way to get prometheus on windows to always announce in the console its data retention effective setting? Thanks in advance!

Sravanthi0614 avatar Feb 16 '23 15:02 Sravanthi0614

I also saw in the console multiple ..."Deleting obsolete block"... lines.

I believe this is normal in that the system is compacting data and/or restructuring the data internally. So it's not removing data really, it's just shifting it around for optimizations.

is there a way to get prometheus on windows to always announce in the console its data retention effective setting?

I believe there's a config page you can navigate to. This may tell you what you're looking for.

johrstrom avatar Feb 16 '23 15:02 johrstrom