moby
moby copied to clipboard
moby
Global .dockerignore
Problem description
Users and developers use different editors and tool to edit, backup and process code, documentation and assets. It is not feasible to catch them all in a project's .dockerignore. Instead, a user should be able to blacklist the file of their own toolchain.
Proposed solution
Add a global .dockerignore, e.g. at $HOME/config/docker/dockerignore, which gets combined with the project's .dockerignore.
References
Docker is not the only tool with this problem. The problem is well known in the VCS world. GIT for example accepts the global core.excludesfile option which points to a global .gitignore. This file is combined with the projects .gitignore to blacklist more files.
Open questions
- Where should the config file be located on Windows?
- Should the global ignore file virtually appended or prepended to the local file? Does this make a huge different? If so, what are the implications and the pros and cons of the different strategies?
Not sure how I feel about this yet, but if we do have a global .dockerignore then it should probably be in the newly created $HOME/.docker config dir.
@duglin Nope, docker should not create another directory directly under $HOME. Instead it should follow the XDG directory specification
:+1: Definitely, in favor a global .dockerignore. Would be very happy if this was similar to the way the global .gitignore file is specified.
+1
This might be an osx specific response...
I use docker-osx-dev for folder sharing in dev as vboxsf is slow.
docker-osx-dev obeys the .dockerignore file.
In an ideal world, .dockerignore would be a symlink to .gitignore. However, this results in the .git directory being synced. So instead, I do:
cp .gitignore .dockerignore
echo '.git' >> .dockerignore
Having a global .dockerignore with .git inside would mean a project's .dockerignore can be a symlink to .gitignore, which would be a lot simpler.
-- edit -- I hadn't appreciated the options available with docker-osx-dev as this command achieves what I explained above:
docker-osx-dev -i .gitignore -e .git
@akash1810 the semantic of .dockerignore is close to but identical to .gitignore. For example /foo doesn't work in .dockerignore. foo means different things.
One use-case for a global .gitignore would be to set os-specific files like the .DS_Store which should not be a concern of the project.
I'm not to familiar with what Docker needs to ignore, but someone requested a new gitignore template for https://www.gitignore.io.
Is there a standard .dockerignore file that I can use for the template?
@joeblau the .dockerignore is specifying which local files should be excluded when building an image (e.g, tmp files etc), however, it really depends on the project (if could be a Java, PHP, Python project), so I doubt it's possible to define a sensible default
@thaJeztah that's what I thought. I feel like the template this person is asking for isn't possible. What they need is a custom template based on the .dockerignore which I could also generate.
Though there are some things though like the Readme, License, maybe .git that are often excluded.
the
.dockerignoreis specifying which local files should be excluded when building an image (e.g, tmp files etc), however, it really depends on the project (if could be a Java, PHP, Python project), so I doubt it's possible to define a sensible default
Disagree. At least two things can be safely ignored globally, as mentioned above:
- OS-dependent temporary files, such as
.DS_Storein macOS, and*~in Linux with gedit. - Version-control folders, such as
.gitof Git,.hgfor Mercurial, and.svnfor Subversion.
Whether the project be a Java, PHP, or Python project, you never want them in the image. Currently the only solution is to duplicate all those common entries in every .dockerignore.
@franklinyu my reply was to "Is there a standard .dockerignore file that I can use for the template?"
Although .git directories are often excluded, there are lots of examples where they are actually needed to build an image. So "safely" is not "safely" in all cases, so cannot be recommended to provide a default template.
I have a custom build system in Sublime text that runs a script in the project directory. This script is where I set up whatever various commands I want to be run when I hit Ctrl+B, and I tweak the script based on whatever I'm working on. It's just for me, and it's highly volatile, so I don't ever want it checked into version control or built into the docker image if I build one.
I have a convention of naming files that are "just for me" with a .jemc suffix so it's obvious to me that they're not meant to be shared, and I have a global .gitignore set up for them. It would be nice if I could do a global .dockerignore for them as well, because we obviously don't want to have to add *.jemc to the .dockerignore for our projects.
Another use case for this is when using environment managers such as direnv. I have .envrc and .direnv in my global .gitignore because .envrc contains my environment variables (potentially containing API keys and other secrets) and .direnv contains pip packages for my editor (which just add useless files to a container). I have to add these two files to ever .dockerignore manually; a single place to manage them would be nice to have.
Another use case is that I (and probably not only I) create .Stuff directory inside every project and I store there any project-related non-commitable things ONLY FOR ME. I don't want to add .Stuff to .dockerignore since this is not project's part, this is only my workflow convention to keep useful things easily accessible (from IDE for example).
With global .gitignore i can ignore those directories but for Docker I have to add it to .dockerignore (which is bad) or change convention (which I don't want).
this would be great, for people who use jetbrains, or whatever creates .idea everywhere. maybe ~/.config/docker/ignore or something?
I miss this also, but I also found that adding .* at the very top of the .dockerignore file works pretty well for local builds. Your mileage may vary, explicit use of ! rules afterwards make certain context requirements more visible, too.
/e: To fully skip context, use stdin for docker build. Maybe building from a local git repo for context works as well, I've never tried it.
I'd also be interested in having updates.
I understand the reasoning about "each project is different".
However, what's globally .dockerignored is a choice up to the host machine owner. Same as with global .gitignore.
Like everything else on their machine, after all.
A team can surely agree on something. But it's unrealistic for a public repo maintainer to address the needs of every single user, OS or environment.
If Git can do it, I (personally) don't see why Docker can't.
@Pictor13 Likely because that are two different pair of shoes (and it's not the question of can or can't do - technically this is obviously trivial, which makes me wonder how it imposes any issues...).
So actually wondering about what drives you with your comment beating a dead horse.
Normally I would put the domain of the build more to the project (VCS/SCM) than to the systems side, so I'm wondering why I should let my system administrator configure that?
And have you considered so far to let git do this? As you write, git can do it, was there a blocker doing it with git?
So actually wondering about what drives you with your comment beating a dead horse.
What you call "dead" has actually no consensus, no clear decision, is still linked by #40319 as potential improvement and is still marked as open.
I just didn't want to ask for news writing a useless "Any updates? It's been 6 years...".
And have you considered so far to let git do this? As you write, git can do it, was there a blocker doing it with git?
I was talking of feasibility, as you also mentioned. A global .gitignore won't prevent locally added files from being sent to the build context.
What you call "dead" has actually no consensus, no clear decision, is still linked by #40319 as potential improvement and is still marked as open.
@Pictor13 Sorry, using an idiom and what I meant is that from the three years or so ago when I was sharing with and contributing my comments to moby, somewhere between up to today, I stopped waiting for that feature and instead moved left and never had to return back - and doing so even if today this feature would be available would be a degradation for me. YMMV, this is with all due respect.
Reading your comment - which was not complaining! - and reading about your knowledge about git - I was wondering whether or not you gave it a try already and hit blockers.
A global .gitignore won't prevent locally added files from being sent to the build context.
That depends a lot how you create the directory tree (or tarball or git repository) you let docker build obtain the build context from. Both horizontally and vertically.
But at the end of the day it's the question whether you want to configure the build in the repository or in the system.
So most likely on the one hand I can imagine it appears not possible with git and on the other hand it could be also that it should not be done with git (perhaps the system then is managed with some git, so also git again ).
Choose your weapon first.
And yes,. please continue to demand that feature. It's very important, I at least once needed it myself, see above. So at least my past self still wants this!
And docker-build really needs a driver for git on the local file-system. So that it does not require network to build a revision from a local, non-bare repository.
For what it's worth, I think this issue should be closed. If that feature were introduced, it would cause even more confusion for users. Now instead of having reproducibility issues because too many files are added, you have the reverse where some users have .ignored too many files. People on issues now have to ask users about their global .gitignore to make sure it didn't interfere with files in the project.
The only way to be reproducible is to list each file that gets added individually.
@zimbatm i disagree, dockerignore should only include artifacts relevant to the project, not cruft only relevant to a particular engineer’s local dev env
I’ve never seen anyone asking about a users gitignore, but the only stuff that should go in there are .DS_Store, .idea, etc. Junk produced by your personal env. what are people doing :(
@quinn-freshly I think zimbatm's concern is that (for example) if a user's global .dockerignore contains .env, but the .env file is needed for the project to build properly, then you will have a whole class of strange bugs. Even if putting .env there is a bad idea, people may still do it and it will be hard to diagnose. This is a good point, but I don't think it means this shouldn't be added at all. (maybe ! .env in a local .dockerignore could override it?)
