Multiline environment variables break the docker-compose bootstrap script

Description

It results in unexpected behavior if either of the environment variables contain newlines (\n). In other words, multiline environment variables break docker-compose startup script. This is especially a painful bug when docker-compose is used in a CI pipeline where webhooks of external applications inject such values. For example, when a commit happens, the Jenkins plugin "GitLab branch sources" injects GITLAB_COMMIT_MESSAGE_XX which contains the commit message which is potentially multiline.

Steps to reproduce the issue:

1. Define a multiline environment variable: export FOO=$(echo -e "foo\n\nbar\n\nbaz"). Confirm that the variable is set and indeed breaks into multiple lines via printenv.
2. run docker-compose

Describe the results you received: Unable to find image 'bar:latest' locally docker: Error response from daemon: pull access denied for bar, repository does not exist or may require 'docker login': denied: requested access to the resource is denied. See 'docker run --help'.

Describe the results you expected: (standard help text of docker-compose)

Additional information you deem important (e.g. issue happens only occasionally):

• One could see how the docker-compose script (located in usr/local/bin) is broken if they run the second step as bash -x docker-compose. concatenating environment variable "keys" with -e does not happen the way it should (empty occurrences of -e)
• I suspect that the script invoking sed at line 74 ("Setup environment variables for compose config and context") is where the bug resides.

Output of docker compose version:

docker-compose version 1.29.2, build 5becea4
docker-py version: 5.0.0
CPython version: 3.7.10
OpenSSL version: OpenSSL 1.1.1k  25 Mar 2021

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 2
  Running: 2
  Paused: 0
  Stopped: 0
 Images: 6
 Server Version: 20.10.8
 Storage Driver: devicemapper
  Pool Name: docker-253:0-204118593-pool
  Pool Blocksize: 65.54kB
  Base Device Size: 10.74GB
  Backing Filesystem: xfs
  Udev Sync Supported: true
  Data file: /dev/loop0
  Metadata file: /dev/loop1
  Data loop file: /var/lib/docker/devicemapper/devicemapper/data
  Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
  Data Space Used: 2.79GB
  Data Space Total: 107.4GB
  Data Space Available: 41.23GB
  Metadata Space Used: 22.24MB
  Metadata Space Total: 2.147GB
  Metadata Space Available: 2.125GB
  Thin Pool Minimum Free Space: 10.74GB
  Deferred Removal Enabled: true
  Deferred Deletion Enabled: true
  Deferred Deleted Device Count: 0
  Library Version: 1.02.170-RHEL7 (2020-03-24)
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b63
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-1160.6.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31.26GiB
 Name: (censored)
 ID: IM5W:HIWL:XL53:AECA:TAGM:ZEQH:76GF:7ZKB:BHLT:SSOS:HHJR:4MWE
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: true
 Insecure Registries:
   (censored)
 Live Restore Enabled: false

WARNING: the devicemapper storage-driver is deprecated, and will be removed in a future release.
WARNING: devicemapper: usage of loopback devices is strongly discouraged for production use.
         Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.

Additional environment details:

• Docker-compose has been installed "as container": see https://docs.docker.com/compose/install// --> Alternative install options --> Install as a container

As a solution I patched the original bootstrap script on the host machine as follows.

In /usr/local/bin/docker-compose:74 I replaced the following line: ENV_OPTIONS=$(printenv | sed -E "/^PATH=.*/d; s/^/-e /g; s/=.*//g; s/\n/ /g") with ENV_OPTIONS=$(printenv --null | sed --null-data -E "/^PATH=.*/d; s/^/ -e /g; s/=.*//g; s/\n/ /g")

The point is that:

• I made printenv use NUL as delimeter (instead of default newline) using --null option (see documentation)
• I instructed sed that separator for "lines" in the incoming stream is NUL using --null-data option (see documentation)
• Prepended -e at index 51 of original line with a space so that the resulting command is syntactically correct.

If your environment setup (for example busybox) does not support either of the options in the above snippet, you should try @lightoze's fix:

https://github.com/docker/compose/issues/9103#issuecomment-1012174647

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

This issue has been automatically closed because it had not recent activity during the stale period.