docker
Escape sequences cannot be used inside quoted strings
Description of the issue
Docker Compose v2 (compose-go) is stripping out escape characters when inside of a quoted string in a command (might happen in other circumstances).
If it's not in a quoted string, they are left as-is. For example, command: printf 'Hello\tworld\n' works while command: sh -c "printf 'Hello\tworld\n'" does not.
Context information (for bug reports)
- [x] Using Compose V2
docker compose ... - [ ] Using Compose V1
docker-compose ...
Output of docker(-)compose version
Docker Compose version v2.0.0-rc.2
Output of docker version
Client:
Cloud integration: 1.0.17
Version: 20.10.8
API version: 1.41
Go version: go1.16.6
Git commit: 3967b7d
Built: Fri Jul 30 19:55:20 2021
OS/Arch: darwin/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.8
API version: 1.41 (minimum version 1.12)
Go version: go1.16.6
Git commit: 75249d8
Built: Fri Jul 30 19:52:10 2021
OS/Arch: linux/amd64
Experimental: true
containerd:
Version: 1.4.9
GitCommit: e25210fe30a0a703442421b0f60afac609f950a3
runc:
Version: 1.0.1
GitCommit: v1.0.1-0-g4144b63
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Output of docker-compose config
w/ v2 (v2.0.0-rc.2):
services:
hello-world:
command:
- sh
- -c
- printf 'Hellotworldn'
image: busybox
networks:
default: null
networks:
default:
name: escape-repro_default
w/ v1 (1.29.2, build 5becea4c):
services:
hello-world:
command: sh -c "printf 'Hello\tworld\n'"
image: busybox
version: '3.9'
Steps to reproduce the issue
- Use an escape sequence (e.g.
\tor\n) inside of a quoted string incommand
Observed result
- Docker Compose v2 (
compose-go) strips out the\characters
Expected result
- Docker Compose leaves the
\characters as-is
Stacktrace / full error message
n/a
Additional information
n/a
compose v2 config do render the fully parsed compose model, as it is used to invoke on engine API and create container. Doing so, command is parsed into a slice of command and arguments.
but getting \t converted into unescaped character t is definitively a bug.
Yeah - the conversion from string -> list isn't surprising, just the stripping out of \ characters.
Output of running under both versions:
v1:
$ docker-compose -p escape-repro -f escape-repro.yml up
Creating network "escape-repro_default" with the default driver
Creating escape-repro_hello-world_1 ... done
Attaching to escape-repro_hello-world_1
hello-world_1 | Hello world
escape-repro_hello-world_1 exited with code 0
v2:
$ docker-compose -p escape-repro -f escape-repro.yml up
[+] Running 2/2
⠿ Network escape-repro_default Created 0.1s
⠿ Container escape-repro_hello-world_1 Created 0.1s
Attaching to hello-world_1
hello-world_1 exited with code 0
(FWIW There's also a subtle difference here in v2 which is that if the last log line doesn't end in \n it is never printed - in v1 it would get printed although it kinda messes with the output since then the "exited with code" line from compose is on the same line as it but 🤷)
It actually works with double backslash.
Ex. \\t and \\n
Looks like the backslash is escaped too.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue has been automatically marked as not stale anymore due to the recent activity.
seems that all my bash scripts that were passed in the command to bash containers broke because of this
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
This issue has been automatically closed because it had not recent activity during the stale period.
This issue has been automatically marked as not stale anymore due to the recent activity.
this bug is caused by shellwords.Parse converting the configured string into a ShellCommand
Oh, nice find! Ironically, I just added a use of that function today (but in e2e code so not likely to be affected)
It'd be nice to publish a Go utility lib (or augment one of our existing ones) that handled a lot of the POSIX-isms our tools rely one on of these days
I recently also stumbled upon that issue and it is currently present in many test setups for TYPO3 and it's extensions.
E.g. here: https://github.com/TYPO3/styleguide/blob/3baee32b70d60fe13741435597053c240ccc3463/Build/testing-docker/docker-compose.yml#L294
The lines
XDEBUG_MODE=\"off\" \
bin/phpunit -c Web/typo3conf/ext/styleguide/Build/FunctionalTests.xml ${EXTRA_TEST_OPTIONS} ${TEST_FILE};
are executed not as one, but two lines and therefore the XDEBUG_MODE variable is not used by the phpunit command afterwards.
I'm using Debian 12 and docker-compose v2.17.0, another user i chatted with who also had problems with that used Ubuntu 23.04 and docker-compose v2.17.2.
Is this something that can and should be fixed in docker-compose or should we rewrite our docker-compose.yml files?
@mmunz command should better not be used with such a long shell script. This will make maintenance painful and subject to bugs like this one. Better move this into a script file you mount as a volume (on copy in a Dockerfile) and pass environment to the container so value get resolved during execution. This also make it easier to run this exact same script standalone outside a Docker container for testing purpose
@ndeloof while i tend to agree, it is imho still a problem that docker-compose v1 and v2 parse these scripts differently.
this is for sure an issue, but we don't have any simple solution here.
docker-compose v1 is coded in python, and relies on python's shlex to parse the command attribute. There's no direct equivalent in Go.
The root issue here is that command has been introduce with the assumption we can strictly reproduce shell script parsing in compose codebase. That makes the tool more convenient to configure for users, but some corner cases exists.