terraform-aws-gitlab-runner icon indicating copy to clipboard operation
terraform-aws-gitlab-runner copied to clipboard

Handling of the New Auto Token Rotation Feature

Open MrGiga opened this issue 1 year ago • 4 comments
trafficstars

Describe the solution you'd like

The new runner authentication method includes an auto-rotation feature (disabled by default). For this feature to function, the expiration interval must be configured within GitLab. Once the interval is set and the time is reached, the Gitlab runner automatically initiate a process to reset the token, with the new token stored in the configuration file. The SSM 'token counter' method will be removed in the upcoming version 8.0.0 of this module (As noted here ). I am proposing that that implementation is not removed and instead is re-used to handle the new auto-rotation feature. Otherwise this module will not be compatible with Gitlab instances that have the auto-rotation feature enabled.

Describe alternatives you've considered

None

Suggest a solution

There are two possible solutions:

First Solution (easiest): Mark the handling of token rotation as out of scope and add a disclaimer in the documentation.

Second Solution: Do not remove the "usage counter" SSM parameter and use that along with the reset-token command. The process would be as follows:

  1. SED config.toml for token_expires_at if it exists move to step 2 otherwise skip.
  2. Check the usage counter, if it is 0 then continue to step 2
  3. Run the reset-token command - https://docs.gitlab.com/runner/commands/#gitlab-runner-reset-token
  4. SED config.toml for Token value
  5. Update token value in SSM Parameter

Additional context

The token rotation handling event should most likely occur during the termination of the runners. In my use case I rotate out the runners every week which results in the latest AMI version being used but that would not be suitable for everyone. Handling a race condition properly is hard.

MrGiga avatar Oct 15 '24 01:10 MrGiga

The first solution is not an option as rotating the tokens adds security.

Solution two sounds feasible.

We have to make sure that older versions of the Runner are still working. Not sure when the reset-token command was added.

kayman-mk avatar Oct 15 '24 08:10 kayman-mk

The first solution is not an option as rotating the tokens adds security.

Solution two sounds feasible.

We have to make sure that older versions of the Runner are still working. Not sure when the reset-token command was added.

Looks like it first appeared in v15.10.0

Sources: MR: https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3209 Commit: https://gitlab.com/gitlab-org/gitlab-runner/-/commit/5e8770d2cb1606f7db37d2ae706c8ad099e57295

MrGiga avatar Oct 15 '24 15:10 MrGiga

@MrGiga Any chance to get a PR from you? This feature looks valuable.

kayman-mk avatar Jan 16 '25 08:01 kayman-mk

@MrGiga Any chance to get a PR from you? This feature looks valuable.

I can take a look. Solving the race condition is pretty intimidating.

MrGiga avatar Feb 10 '25 19:02 MrGiga