invalid variable names in docker/elasticsearch/env/docker.env when using docker compose v2

[nikmartin]

Installing the latest docker compose (v2) will cause the dev.sh script to throw errors:

unexpected character "." in variable name near "discovery.type=single-node\nxpack.security.enabled=false\nES_JAVA_OPTS=-Xms256m -Xmx256m -Dlog4j2.formatMsgNoLookups=true\n"

[pedro93]

@dexter-mh-lee could you take a look at this please?

@nikmartin are you using a custom docker-compose file? Where are you getting it from? Which version of DataHub are you trying to deploy?

[nikmartin]

Am using https://github.com/linkedin/datahub/blob/master/docker/docker-compose-without-neo4j.yml, and the env file for elastic search is the offending file. The issue is most definitely with docker compose v2, as switching back to compose v1 works ok.

[dexter-mh-lee]

@nikmartin Seems like we need to put quotes around variables. Can you try putting quotes around the value side of this variable and see if that works with docker compose v2? https://github.com/linkedin/datahub/blob/master/docker/elasticsearch/env/docker.env#L3 Feel free to put out a PR with the change if that solves it!

[nikmartin]

I didn't mention it in my original issue, but I had done a bit of research and saw a few posts about similar issues, but the issue was the '.' in the key name, and no amount or flavor of escaping could get it to pass. When I deleted lines 1&2 and wrapped line 3 in quotes, it worked ok.

[dexter-mh-lee]

So ES_JAVA_OPTS="-Xms256m -Xmx256m -Dlog4j2.formatMsgNoLookups=true" doesn't work?

[nikmartin]

yes that works fine, but the first two lines in the default env file have "." In the key name. Those I can't get escaped: ​discovery.type=single-node

[pedro93]

@nikmartin I'm very sorry for not replying sooner! Could you please share the command you are using to launch DataHub and your version of docker-compose so I can try to repo locally?

[yactouatfuture]

hello everyone ! I had the same issue than @nikmartin just after having downloaded and trying to run https://github.com/temporalio/docker-compose , as it happened running docker-compose up instead of docker compose up solved it, hope that helps

[vincent-lecrubier-skydio]

On macos, I had to run docker-compose disable-v2 so that my docker-compose didn't run on v2. See https://docs.docker.com/compose/cli-command/#install-on-mac-and-windows

[dexter-mh-lee]

@pedro93 has a PR out with a fix! This should be fixed by next release

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

[github-actions[bot]]

This issue was closed because it has been inactive for 30 days since being marked as stale.