Command value wrapped in quotes is not interpreted correctly

[barseghyanartur]

ATM, constructions like command: '"0 0 * * * gosu sentry sentry cleanup --days $SENTRY_EVENT_RETENTION_DAYS"' are not interpreted correctly by rec_subs function. The value of the command after going through rec_subs would be '"0 0 * * * gosu sentry sentry cleanup --days'.

Taken from Sentry OnPremise repository:

  sentry-cleanup:
    << : *sentry_defaults
    image: sentry-cleanup-onpremise-local
    build:
      context: ./cron
      args:
        BASE_IMAGE: 'sentry-onpremise-local'
    command: '"0 0 * * * gosu sentry sentry cleanup --days $SENTRY_EVENT_RETENTION_DAYS"'

I kind of fixed that in my fork in a very simplistic way. Please, let me know if you think it's good enough (so that I submit a PR).

[rmarting]

I have the same issue with the following docker-compose.yml file to pass environment variables in one of the command in my environment:

version: '3'
services:
  zookeeper:
    image: strimzi/kafka:0.20.0-kafka-2.5.0
    command: [
      "sh", "-c",
      "bin/zookeeper-server-start.sh config/zookeeper.properties"
    ]
    ports:
      - "2181:2181"
    environment:
      LOG_DIR: /tmp/logs
      
  kafka:
    image: strimzi/kafka:0.20.0-kafka-2.5.0
    command: [
      "sh", "-c",
      "bin/kafka-server-start.sh config/server.properties --override listeners=$${KAFKA_LISTENERS} --override advertised.listeners=$${KAFKA_ADVERTISED_LISTENERS} --override zookeeper.connect=$${KAFKA_ZOOKEEPER_CONNECT}"
    ]
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
    environment:
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      LOG_DIR: /tmp/logs

When I started the pods the kafka container fails because the start up command is:

bin/kafka-server-start.sh config/server.properties --override listeners=$ --override advertised.listeners=$ --override zookeeper.connect=$

Instead of:

bin/kafka-server-start.sh config/server.properties --override listeners=PLAINTEXT://0.0.0.0:9092 --override advertised.listeners=PLAINTEXT://localhost:9092 --override zookeeper.connect=zookeeper:2181

It should be nice to have this bug resolved in podman-compose to allow this kind of set up in containers. I did not tried the fork provided by @barseghyanartur, but it could be nice to be accepted.

Is there a workaround to resolve it?

My environment is:

ws-streams/blue-green/src 
❯ cat /etc/redhat-release 
Fedora release 32 (Thirty Two)
❯ podman-compose -f docker-compose.yml version
podman-composer version  0.1.6dev
podman --version
podman version 2.1.1

[muayyad-alsadi]

it seems that your problem is with .env as the following part --override listeners=$${KAFKA_LISTENERS} should read ${KAFKA_LISTENERS} from .env which have the value of PLAINTEXT://0.0.0.0:9092. the question is did you copy your .env?

[rmarting]

Sorry @muayyad-alsadi but I cannot follow your question. What do you mean with "copy your .env"?

The environment variables in the container are already declared by environment section:

sh-4.2$ echo $KAFKA_LISTENERS 
PLAINTEXT://0.0.0.0:9092

But in the command line these variables are not replaced with the right values.

[muayyad-alsadi]

please read this

https://docs.docker.com/compose/environment-variables/

the variables inside docker-compose are not read from shell environment variables but from .env file in the current working directory.

[rmarting]

Thanks @muayyad-alsadi for the reference. I was wrong and now it is working fine!

[pilhuhn]

@rmarting How did you solve it (except for hard coding the values in the command)? @muayyad-alsadi I think the way @rmarting has described it is the way the Srimzi folks (used to) describe the setup with docker-compose (where I had that successfully running for ages). In that case I would also expect that podman-compose can deal with it.

[kevinkjt2000]

Double $$ should escape the $ symbol; instead, what's happening now is that $${VARIABLE} is inserting $value of variable from .env file. The expected behavior is for the result of $${VARIABLE} to be ${VARIABLE}.

There's a note about using $$ here https://docs.docker.com/compose/compose-file/compose-file-v3/#variable-substitution

[muayyad-alsadi]

@kevinkjt2000 this is supposed to be fixed by

https://github.com/containers/podman-compose/pull/97

are you sure you are using latest devel branch?

we have a test for this tests/interpolation/docker-compose.yml

EXAMPLE_LITERAL: This is a $$literal

and it gives the desired result which is $literal as in the spec document you have just mentioned

@pilhuhn should we close this issue as it seems working fine?

[kevinkjt2000]

are you sure you are using latest devel branch?

Of course, I was not. As with any piece of new software I opt for using the latest stable released version that comes by default from a package manager. You have given me a good reason to step away from that default and take on some level of extra effort to get the development version. At least, that would have been the case if I hadn't already reverted to using docker-compose because podman machine networking is giving me trouble on my Macbook, but that's outside the scope of this issue.

I see that #97 was merged months ago. It would be nice to see this released so others don't keep wasting their time finding this issue again like I did.