gitlab-migration icon indicating copy to clipboard operation
gitlab-migration copied to clipboard

Added Dockerfile as a fixed runtime env

Open brain-slug opened this issue 4 years ago • 4 comments

Added Dockerfile, updated IFS from CRLF to LF only, renamed access token vars allowing to be set from Dockerfile, added CURL_PARAMS --raw to avoid encoding issues, updated README.md

Fixes #7

brain-slug avatar Oct 11 '21 12:10 brain-slug

Hi @brain-slug, thank you for your pull request. I'll need to test your changes locally first as I'm not sure why you would need to set the --raw parameter at all. I have the exact same versions of jq, curl, bash on my local WSL2 setup, so it must be something in the Docker Base image that's interferring. I'll test it early November when I'm back in the office.

albuch avatar Oct 12 '21 08:10 albuch

Hi @albuch I am pretty sure it is due to encoding issues. By default the ubuntu base image comes with the following locale setting:

LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

while the migrate.sh script set local LANG=C i c e=''for the urlencode() function. So setting curl --raw should work around this OS-specific issue without breaking it on windows and linux.

If undesired I could imagine adding some code for the locale/encode/collate discovery and set it accordingly (same applies to the CRLF vs LF issue).

brain-slug avatar Oct 12 '21 11:10 brain-slug

From what I can tell, the issue comes from LANG not being set in your Docker Image. On the environments that I tested it is set to LANG=C.UTF-8. Group and Project names can have UTF-8 characters and you should make sure that your shell can handle these. Please remove --raw parameter and set an oppropriate LANG value in the Docker image instead along with fixing the other review remarks if you want this PR to move forward.

albuch avatar Mar 05 '22 14:03 albuch

Unfortunately this did not fix the issue. Ubuntu base image ships with C.UTF-8 so I actually tried setting the ENV in the Dockerfile removing the CURL_PARAMS --raw:

ENV LANG C.UTF-8
ENV LANGUAGE C.UTF-8
ENV LC_ALL C.UTF-8

and again got the same error like before: curl: (3) URL using bad/illegal format or missing URL

brain-slug avatar Mar 07 '22 10:03 brain-slug