Documentation section on the escape character needs an update

[ferrlen]

File: engine/reference/builder.md

Hi!

I believe the section https://docs.docker.com/engine/reference/builder/#escape is outdated and should be rewritten.

First, microsoft/nanoserver is not maintained anymore, so running FROM microsoft/nanoserver [implicit tag: latest] will output an error. But most importantly, I couldn't reproduce the COPY testfile.txt c:\\ bug mentioned there, this has probably been fixed already. I only get a weird bug if I run RUN dir c:\\, which is absent if RUN dir c:\somedir\\ is run instead.

I'd like to ask one of the mods/people in charge to please update this section, so other users don't waste time trying to figure out how to escape the "escape character". I think it can always be escaped by using it twice (for example: \\), except in RUN or CMD instructions

Thanks

[thaJeztah]

I'm not running Docker on Windows myself; do you have more details on;

• what image you're running instead of microsoft/nanoserver (wondering if there's a difference in the image config)
• what error you're getting with the "I only get a weird bug"?

Perhaps you can post exact steps what you used, and also post docker version and docker info output.

[ferrlen]

Yes, to answer each question:

1. According to their instructions (changed recently to 2022), I ran instead FROM mcr.microsoft.com/windows/nanoserver:win10-21h1-preview ("Note: This repo does not publish or maintain a latest tag. Please declare a specific tag when pulling or referencing images from this repo.")
2. I've since uninstalled the whole set up, thus I'm not able to run it again and get docker info. But the weird bug I referenced is that RUN dir c:\\ produces an error, because the ending \\ first escapes the second backslash, then escapes newline (\n). For example:

...
RUN dir c:\\
RUN someapp.exe

Would actually be parsed as:

...
RUN dic c:RUN someapp.exe

You'd expect that a similar instruction, such as RUN dir c:\somedir\\, would run into the same bug. However, it's interpreted correctly, as RUN dir c:\somedir without escaping the trailing newline character. That's why I called it weird, I don't know why the first instruction is misread, but the second isn't.

In terms of Docker version, I can't remember whether I used Docker 4.8.2 or 4.9, I believe I used 4.9.

Here's a fuller example:

# escape=\

FROM mcr.microsoft.com/windows/nanoserver:win10-21h1-preview
# the line below does NOT produce an error
COPY testfile.txt c:\\
# the line below WILL produce an error, this step reads as: `RUN dir c:RUN dir c:\somedir`
RUN dir c:\\
# however, the steps below run fine, once we eliminate the problematic line above 
RUN dir c:\somedir\\
RUN dir c:\somedir\subdir\\

[thaJeztah]

According to their instructions (changed recently to 2022), I ran instead FROM mcr.microsoft.com/windows/nanoserver:win10-21h1-preview ("Note: This repo does not publish or maintain a latest tag. Please declare a specific tag when pulling or referencing images from this repo.")

Yes, we need to find a good replacement image for the examples in our documentation. Unfortunately, the Windows images generally need to have a base-image that matches the kernel version of the machine they're run on, and Microsoft unfortunately decided not provide multi-arch images. This makes it a bit complicated to have examples in our documentation that "work for everyone". 🤔

the weird bug I referenced is that RUN dir c:\ produces an error, because the ending \ first escapes the second backslash, then escapes newline (\n). For example:

Right, so that issue is related to the warning that you pointed to; the trailing \ will be seen as a line-continuation marker when the Dockerfile is parsed, which is a common pattern, e.g. both RUN lines below are equivalent;

RUN echo hello \
world

RUN echo hello world

This, of course, is problematic with Windows, which uses \ as path-separator, which is why the escape directive was added to allow using a different symbol (`).

However, it's interpreted correctly, as RUN dir c:\somedir without escaping the trailing newline character. That's why I called it weird, I don't know why the first instruction is misread, but the second isn't.

That's a good question; from a quick look at the example, perhaps it's because the \\ is preceded by a colon (:), and therefore possibly handled by a different code-path. The COPY and RUN commands may also hit different code-paths.

I recall we made some improvements to make it "work" for more cases (see https://github.com/moby/buildkit/pull/1559), but in general, the recommendation would still be to use the escape directive if the dockerfile contains trailing backslashes and therefore ambiguous.

That said; we could improve the examples if needed (contributions welcome!); we may have to do so after https://github.com/moby/buildkit/pull/2902 is merged though, as we're in the process of moving the Dockerfile reference documentation to the BuildKit repository (which is where the code for parsing the file is maintained).

[ferrlen]

Thank you for your prompt reply!

For the time being, I think just the parts of the example that are wrong could be updated immediately.

I agree with you that it's a nuisance that Microsoft isn't providing a latest tag, but maybe then the solution is to pick a fairly stable old image, which can be a new image today, like ltsc2022. Thus, one would only need to update FROM microsoft/nanoserver to FROM microsoft/nanoserver:ltsc2022.

In regard to the 2nd issue, I still don't know why that happens, but the immediate solution might be just to update the example eliminating the COPY testfile.txt c:\\ line, since it doesn't run into any bugs (and then also update the explanation that incorrectly claims that a bug would be produced).

For the long run, it'd be a good idea to explain how to escape the escape character. From my tests, using it twice (like \\) works every time (like in most languages), except for the RUN/CMD instructions. Changing the escape character might by itself bring in new headaches, maybe it's just me but I usually opt to just go for the default