Got error on `Assemble new config.toml`

[ishkulov]

Running with config as described in example got an error FATAL: Near line 26 (last key parsed 'runners.docker.sysctls'): bare keys cannot contain '.'

Message:

ASK [riemers.gitlab-runner : Assemble new config.toml] ************************
fatal: [default]: FAILED! => {"changed": false, "msg": "failed to validate: rc:1 error:Runtime platform                                  \u001b[0;m  arch\u001b[0;m=amd64 os\u001b[0;m=linux pid\u001b[0;m=12602 revision\u001b[0;m=0e5417a3 version\u001b[0;m=12.0.1\nRunning in system-mode.                           \u001b[0;m \n                                                  \u001b[0;m \n\u001b[31;1mFATAL: Near line 26 (last key parsed 'runners.docker.sysctls'): bare keys cannot contain '.'\u001b[0;m \n"}

[GMartinez-Sisti]

I'm having the same error, but I know the reason why this is happening.

I wanted to add the following keys:

builds_dir: "{{ gitlab_data_builds }}"
cache_dir: "{{ gitlab_data_cache }}"

So I went from this:

gitlab_runner_registration_token: "{{ gitlab_shared_token }}"
gitlab_runner_runners:
    - name: "{{ ansible_hostname }}"
    state: present
    executor: docker
    concurrent_specific: "{{ (ansible_processor_vcpus * 0.6) | int }}"
    docker_image: "ubuntu:16.04"
    tags:
        - docker
    run_untagged: true
    docker_privileged: false
    extra_configs:
        runners.docker:
            memory: "{{ (ansible_memtotal_mb / 2) | int }}m"
            cpus: "{{ (ansible_processor_vcpus / 2) | int }}"

to this:

gitlab_runner_registration_token: "{{ gitlab_shared_token }}"
gitlab_runner_runners:
    - name: "{{ ansible_hostname }}"
    state: present
    executor: docker
    concurrent_specific: "{{ (ansible_processor_vcpus * 0.6) | int }}"
    docker_image: "ubuntu:16.04"
    tags:
        - docker
    run_untagged: true
    docker_privileged: false
    builds_dir: "{{ gitlab_data_builds }}"
    cache_dir: "{{ gitlab_data_cache }}"
    extra_configs:
        runners.docker:
            memory: "{{ (ansible_memtotal_mb / 2) | int }}m"
            cpus: "{{ (ansible_processor_vcpus / 2) | int }}"

However, the change above didn't change the config.toml file after applying the role, so I tried:

gitlab_runner_registration_token: "{{ gitlab_shared_token }}"
gitlab_runner_runners:
    - name: "{{ ansible_hostname }}"
    state: present
    executor: docker
    concurrent_specific: "{{ (ansible_processor_vcpus * 0.6) | int }}"
    docker_image: "ubuntu:16.04"
    tags:
        - docker
    run_untagged: true
    docker_privileged: false
    extra_configs:
        runners:
            builds_dir: "{{ gitlab_data_builds }}"
            cache_dir: "{{ gitlab_data_cache }}"
        runners.docker:
            memory: "{{ (ansible_memtotal_mb / 2) | int }}m"
            cpus: "{{ (ansible_processor_vcpus / 2) | int }}"

And got the error:

TASK [riemers.gitlab-runner : Assemble new config.toml] ************************************************************************************************************************************************************************************
fatal: [vm]: FAILED! => {"changed": false, "msg": "failed to validate: rc:1 error:Runtime platform                                  \u001b[0;m  arch\u001b[0;m=amd64 os\u001b[0;m=linux pid\u001b[0;m=29694 revision\u001b[0;m=3001a600 version\u001b[0;m=11.10.0\nRunning in system-mode.                           \u001b[0;m \n                                                  \u001b[0;m \n\u001b[31;1mFATAL: Near line 27 (last key parsed ''): Key 'runners' has already been defined.\u001b[0;m \n"}

Is there any correct way to add these keys on an existing runner? Or do we need to add some logic to it?

[GMartinez-Sisti]

@ishkulov sorry to piggyback on your issue.

How does your config look like? From the error looks like you might be using:

extra_configs:
        runners.docker.systcl:

Have you tried?

extra_configs:
        runners.docker:
            systcl: "options"

[riemers]

@ishkulov can you check if this solves your issue?

[ishkulov]

@GMartinez-Sisti I used example from https://galaxy.ansible.com/riemers/gitlab-runner.

gitlab_runner_registration_token: '***'
gitlab_runner_runners:
    - name: 'Lab50 Runner'
      executor: docker
      docker_image: 'alpine'
      tags:
          - openjdk 
          - maven 
          - lab50 
      docker_volumes:
          - "/var/run/docker.sock:/var/run/docker.sock"
          - "/cache"
      extra_configs:
          runners.docker:
              memory: 1024m
              allowed_images: ["maven:*", "openjdk:*"]
          runners.docker.sysctls:
              net.ipv4.ip_forward: "1"

[ishkulov]

With GMartinez-Sisti:add_options i got this:

TASK [ansible-gitlab-runner : Ensure directories] ******************************
fatal: [default]: FAILED! => {"msg": "'dict object' has no attribute 'builds_dir'"}

[GMartinez-Sisti]

@ishkulov The issue above is fixed.

But I'm not sure my PR will fix your initial issue though.

[ishkulov]

@GMartinez-Sisti Yes, previous error is gone but initial is still here.

May be it cat help. There is the file that can't pass validation:

# cat /tmp/ansible.o22yqfgitlab-runner-config/gitlab-runner.1.1YXB_n
  name = "Test Runner"
  url = "https://gitlab.com/ci"
  token = "****"
  executor = "docker"
  [runners.docker]
    allowed_images = ["maven:*", "openjdk:*"]
    memory = "1024m"
    tls_verify = false
    image = "alpine"
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
    shm_size = 0
  [runners.cache]
    [runners.docker.sysctls]
      net.ipv4.ip_forward = "1"

"net.ipv4.ip_forward" should have quotes i think. And should be in [runners.docker] section not in [runners.cache]. https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnersdocker-section

[nfk]

Hi, I have the same issue to override the parameter output_limit from [[runners]] there is a way to do that with extra_configs or should I create a MR to support that param? Does the project support all parameters defined on gitlab doc?

[riemers]

well i presume there are some more, but if nobody needed it before it wasn't there. It started for me also as a "i need x and y" and now it grew to this size, lots of people create PR's and i just maintain it for the community. So yes PR's are always good

[nfk]

Ok, it makes sense. Thank you for your work. I will try to make a PR soon about my issue.

[vitkhab]

There is workaround. It's not great solution and might not work in complex cases. That's why it's not a PR.

Add quotes to both sides of {{ line }} variable in file line-config-runner.yml in tasks. Changing

    regexp: '^(\s*){{ line|regex_escape }} ='
    line: '{{ "  " * (section.split(".")|length) }}{{ line }} = {{ gitlab_runner.extra_configs[section][line] | to_json }}'

to this

    regexp: '^(\s*)"{{ line|regex_escape }}" ='
    line: '{{ "  " * (section.split(".")|length) }}"{{ line }}" = {{ gitlab_runner.extra_configs[section][line] | to_json }}'

[cwoebker]

I received the same error as @ishkulov when using the example from the Readme file. Commenting out / Removing the runners.docker.sysctls part fixed it for me. And net.ipv4.ip_forward was still set to true in the config.toml; by default I assume.

gitlab_runner_registration_token: '***'
gitlab_runner_runners:
  - name: 'Example Docker GitLab Runner'
    executor: docker
    docker_image: 'alpine'
    tags:
      - node
      - ruby
      - mysql
    docker_volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "/cache"
    extra_configs:
      runners.docker:
        memory: 512m
        allowed_images: ["ruby:*", "python:*", "php:*"]
    # runners.docker.sysctls:
    #   net.ipv4.ip_forward: "1"

[jverger]

Hello, i've found a workaround (BUT, there is always a BUT...). Here is my extra_configs section

    extra_configs:
      runners.docker:
        memory: 512m
        allowed_images: ["ruby:*", "python:*", "php:*", "busybox:*"]
      runners.docker.sysctls:
        '"net.ipv4.ip_forward"': "1"

it works when i create the runner BUT, if i remove the runners.docker.sysctls section and i launch again the section is not removed in the runner's config.toml file

Hope someone will dig to how to remove the runners.docker.sysctls section if needed in future updates

[Apfelwurm]

Hi together, i have the same Problem, even with a relative minimal variable definition:

gitlab_runner_runners:
  - name: 'deb1-docker'
    token: '************'
    url: 'https://gitlab.*****/'
    executor: docker

it also happens when i have a bit more configured:

gitlab_runner_runners:
  - name: 'deb1-docker'
    token: '************'
    url: 'https://gitlab.*****/'
    executor: docker
    output_limit: '163840'
    concurrent_specific: '1'
    docker_image: 'debian:buster'
    docker_privileged: 'true'
    docker_tlsverify: false
    docker_dns: '10.10.10.254'
    docker_dns_search: '*****'
    docker_volumes:
      - '/srv/build/src:/srv/src'
      - '/srv/build/ccache:/srv/ccache'
      - '/certs/client'
      - '/cache'
      - '/srv/build/prodbuild:/srv/build/prodbuild'

it also happens when i use the example variable definition from the readme with an adjusted url and token.

The variables are setted like they shoult but the ansible play breaks ofc.

I'm running ansible-playbook 2.7.7 on debian buster and the manually verificytion via /usr/bin/gitlab-runner verify -c /etc/gitlab-runner/config.toml works like intended.

Can someone help me here ? :)

[github-actions[bot]]

Seems this message did not get a lot of love. This does not mean it was not seen but time wise might not have made it to proper attention. This is just the clean up action ;)