prometheus
prometheus copied to clipboard
prometheus
Add instructions how to persist prometheus docker container data
I am not sure if this should be covered here or in the docs repository directly. Additionally, chmod 777 is a security issue.
I am not sure if this should be covered here or in the docs repository directly. Additionally, chmod 777 is a security issue.
I am ok with removing chmod 777 instruction and keep only chown 65534:65534. Or adding some text stating that chmod 777 is a security issue and is not actually recommended but only as a workaround. The reason i included this chmod 777 is that user and group id 65534 seems to me being a magic number that can be changed later so the instruction might fail for later versions.
As i have created this pull request, in my opinion having these suggestions would be very helpful for users (i wasted lots of time and my production data because of not having these suggestions) and many projects consider it being a good helpful behavior to cover the question of data persistency in the official docker image doc. But someone should make a decision to include this or not and i guess it should be the maintainers of the repo. So someone needs to be sure and ask for corrections, approve or reject.
So when i first started using prometheus docker image i had an assumption that i made everything to persist my data. After restarting the server i found out i lost all my metrics for the last two months. I started to search for data persistency instructions and all i found was the answer on StackOverflow https://stackoverflow.com/a/50009322/942521. The answer suggests to use a volume that was discovered only by looking into the prometheus dockerfile. Quote: Surprisingly is not mentioned in the image docs.
Probably keeping only instructions on using a named volume would be enough. In my opinion the main thing is to let users know that there IS a volume that can be used for data persistency.
Probably keeping only instructions on using a named volume would be enough. In my opinion the main thing is to let users know that there IS a volume that can be used for data persistency.
Agree, @shoce . Overall, I like how they've documented it in Grafana: https://grafana.com/docs/grafana/latest/administration/configure-docker/#run-grafana-container-with-persistent-storage-recommended , but I think it would also be enough to just include the first part (using named volumes). In any case, it's probably best not to mention 777 :)
but I think it would also be enough to just include the first part (using named volumes)
@danmichaelo Great, i have updated the diff keeping only the first part! It looks good for me, please let me know if any other updates can be made to make it better.
As a comment to this: "In my opinion the main thing is to let users know that there IS a volume that can be used for data persistency."
I found it frustrating that this was just stated, without any explanation neither here https://prometheus.io/docs/prometheus/latest/installation/#using-docker, nor here https://hub.docker.com/r/prom/prometheus/. I then literally had to Google it and found this Stackoverflow-question: https://stackoverflow.com/questions/50009065/how-to-persist-data-in-prometheus-running-in-a-docker-container.
The storage dir/volumn config is pretty important and basic config, IMHO. It would have been great with some testing example, and then explain that for actual production you should consider a proper volume.
So does using a named volume make it uneccesary to do the whole chown 65534:65534? I found I needed to do that when using -v /my/data/dir:/prometheus:z. Note: I'm using podman and rhel8
So does using a named volume make it uneccesary to do the whole
chown 65534:65534?
I believe so - I didn't have to touch the permissions by using -v prometheus-data:/prometheus
I also had to go hunting on StackOverflow for the answer to a very simple question that should have been answered by the docs. Why hasn't this been merged yet?
This was super helpful but took me a while to find. Would be great if this was merged and added to the docs.
How long are you going to hide this information from prometheus users?! It takes a lot of time for people to find out how to persist data. This merge can save lots of time for users! Why is it held for so long?!
@roidelapluie @danmichaelo
I just found out about this after losing my data for the sixth or seventh time, from an answer at Stack Overflow. How in the name of everything respectful and well-documented on the internet is this still a problem? It's beyond absurd to continue to leave this unaddressed, it's one fact with very little mental lift. Let's do it, yeah?
I feel like it's an inside joke by now, and the devs are just laughing at us. The work has been done, it's one click.