Yacht
Yacht copied to clipboard
SelfhostedPro
[Bug Report] ENV Variables Set in Dockerfile/Image Persist Even When They're Changed In Future Image Updates
Describe the bug Yacht conflates ENV variables set by the image with ENV variables explicitly defined by the user; ultimately treating them as one and the same.
The end result is that if an image is originally pulled that uses an ENV variable, because that value is stored as an ENV variable in Yacht the same as if a user had entered it, the value of that variable persists in all future image updates. This is especially problematic if that image ever changes the value of an ENV variable because that means we'll be using the wrong value going forward. This can easily lead to container failure/corruption as was the case seen with the Linuxserver Calibre image update used in the example below.
To Reproduce Steps to reproduce the behavior:
- Pull linuxserver/calibre:v5.16.0-ls109 and run the container.
- Inspect the container's ENV variables. Note that HOME=/root. This was set in the image.
- Update the container to use linuxserver/calibre:latest and restart the container.
- Inspect the container and note that the ENV variables. Note that you still have HOME=/root even though the image has specified the ENV variable to be HOME=/config. Also note that if you had just pulled the latest tag originally (creating a container from scratch and ignoring steps 1 and 2 above), the ENV variable would of course be HOME=/config.
Expected behavior Yacht should not conflate ENV variables set by the image with ENV variables explicitly defined by the user. Yacht should only store ENV values set by the user. It's okay to display all of the ENV vars and settings, just don't store non-user-entered values.
Screenshots After step 2 above with the original image pull using linuxserver/calibre:v5.16.0-ls109:
After step 4, when updating the container to use linuxserver/calibre:latest) we're stuck with the same value (which I won't repeat here with a screenshot).
Compare this to when we create a new container from scratch with linuxserver/calibre:latest:
Desktop (please complete the following information):
- OS: Ubuntu
- Browser: Edge
- Yacht Version v0.0.6-alpha-2021-02-25--15
Additional context This is the same problem that Portainer suffers from. You can see the following for reference/suggested behavior fixes: Option to use defaults from new image when using duplicate/edit or recreate ENV variables set in Dockerfile are preserved even if they change
** Logs ** N/A
There is unfortunately no way to tell which variables are set by the user and which ones are set in the image. The docker API treats them all the same. I could possibly accomplish this with labels but it would be hard to offer a consistent experience.
There is no information in Yacht not pulled directly from the docker API in relation to containers as that would cause consistency issues when working with containers not created in Yacht. I believe this issue should be created on dockers repository (or docker-py).
Yacht is where I enter the values for user-defined ENV variables. Can Yacht not flag these rows as user-defined?
It cannot as the docker API provides no way to supply that information. This is all I can give the API:
https://docker-py.readthedocs.io/en/stable/containers.html
environment (dict or list) – Environment variables to set inside the container, as a dictionary or a list of strings in the format ["SOMEVARIABLE=xxx"].
I have opened an issue with docker about this to see if it's something they would consider. If not I can see about flagging these via labels but those would then be editable if you modify a container unless specifically barred from modification (which is possible but could still provide an inconsistent experience, I know that this issue also causes an inconsistency in the experience but if docker could add this as a feature it would be the preferred way to go)
I get that, but that's from retroactively pulling the information after the container has already been spun up.
What I'm saying is that with Yacht, you control the front end, you know what users are entering. If users are maintaining their containers in Yacht (which presumably is the purpose of Yacht afterall), then we have the unique ability to "remember"/store which ENV variables are user-defined.
I also understand your point about if users have not maintained the image in Yacht. In such a scenario, I think it's entirely reasonable to expect users to have to add those ENV variables manually (at which point they would then become user-defined).
I don't think there's any way for docker-py to accomplish this on their end since Docker doesn't even specify such a thing in their JSON array with docker inspect. Therefore, only "we" could accomplish it on Yacht by capturing that information upfront.
Thanks for the speedy response, btw!
Yeah, I can do that with port labels. I understand that docker-py doesn't have the ability to do anything but seeing as it's a docker project I'm hoping the right people will see it. In the meantime I'll work on dynamically setting it as a label (only way to store said info in containers and I really don't want app info in the DB).
I'd also be happy to coordinate testing this functionality with you if you're interested via discord to make sure it's a suitable fix: https://discord.gg/KpKutvC
Just @ me if you join and I'll pm you once I've got something going.
Thought of something else you should be aware of in your design solution...
I'm not sure how exactly you envision using labels to solve this, but I should probably warn you that if these "labels" are the same as the traditional Docker container labels...they too have the same problem as ENV variables, where they're set once at the first container creation and then we're stuck with them forever (not updating).
The labels are specific to Yacht. If you look at how Port Labels work now you can see an example. It will be a label like yacht.user.env.VARIABLE_NAME=VARIABLE_VALUE and is able to be modified.
The labels are specific to Yacht. If you look at how Port Labels work now you can see an example. It will be a label like yacht.user.env.VARIABLE_NAME=VARIABLE_VALUE and is able to be modified.
Ah, I see where you're going with this. That's a clever workaround that keeps you from having to store this in the db and instead allows you to keep it bundled with the container itself. I'm impressed.
It's the only place where you can store data from the run command and although it may not be made for it, it solves the problem pretty well. I would be doing a lot more with them if they could be modified while the container is running.