✨ Optional stripping of trailing newline when passing secrets

[yann-soubeyrand]

What are you trying to do?

I try to pass a GitHub token as a secret to a Dagger function which uses it as an environment variable in a container. For this, I use the cmd:gh auth token syntax. The problem is that the gh auth token command issues a trailing newline which is not stripped and appears in the environment variable value. The command using this environment variable doesn’t work then. I’d like to have a way to tell Dagger to strip the trailing newline.

Why is this important to you?

Errors which occur due to the trailing newline may not be obvious to debug and having a documented syntax which works in all the situations to strip this newline could improve the situation.

How are you currently working around this?

Currently, I pass the token like this cmd:echo -n $(gh auth token) to strip the trailing newline.

[yann-soubeyrand]

On Discord, there have been some proposals:

• We could add an argument when passing a secret, like cmd:command-to-run:s.
• We could use a different scheme, like cmd-s:command-to-run.
• We could add an annotation on the function parameter, like in Go

func (*Module) Function(
// +strip
secret *dagger.Secret
)

[yann-soubeyrand]

If feasible, I personally prefer the last one.

[helderco]

If feasible, I personally prefer the last one.

I don't think that's a good option. The function shouldn't need to know if the secret needs stripping or not. That responsibility should fall to the one loading the secret. See also:

I don't think we should do this with options at the point of use. Secrets should be marked to do this at point of load.

- Originally posted by @jedevc in https://github.com/dagger/dagger/issues/7220#issuecomment-2090641708

I also like @sipsma's suggestion here:

Hm yeah I agree this is likely what users would want 99.99% of the time and fairly annoying to require them to deal with themselves in userspace (i.e. get secret plaintext, strip newlines, create new modified secret).

I feel like there must be some obscure corner case where this isn't desired, but I can't think of any concretely either. And push comes to shove if we need to support the 0.01% of cases we can always add some kludge like --token=file-raw:/some/token/file/where/trailing/newlines/matter or similar.

- Originally posted by @sipsma in https://github.com/dagger/dagger/pull/6845#pullrequestreview-1920691440

So if the most common thing is to want to strip the newline, then we strip it by default, and offer these alternatives to keep it as is:

• env-raw:
• file-raw:
• cmd-raw:

Which seems clearer than the opposite default:

• env-strip:
• file-strip:
• cmd-strip:

Flipping defaults is always a breaking change though, and @sipsma already reverted his suggestion to default to not touch it:

I'm good with reverting the change to strip newlines. I think we could consider the inverse of my suggestion on that PR and instead support a CLI syntax that does strip newlines for use cases that call for that. But I think this situation proves it's probably better to default to the exact contents of the file/command/env var.

- Originally posted by @sipsma in https://github.com/dagger/dagger/issues/7220#issuecomment-2094405767

Emphasis mine, because I tend to agree with that.

[cbochs]

This has become more present of an issue since the introduction of .env files. From the command line it's possible to do follow the workaround of cmd:echo -n $(gh auth token), but in an env file this kind of substitution raises an error

! load user defaults: failed to load outer env file from ".env": GITHUB_TOKEN: unexpected command substitution at 1:17

Where

GITHUB_TOKEN="cmd:echo -n $(gh auth token)"

I've tried other variations, but have yet to find a working solution:

# Env file parsing removes the "'"s and the command does not execute as intended (the gh help is printed)
GITHUB_TOKEN="cmd:sh -c 'gh auth token | tr -d \\n'"

# Parses GITHUB_TOKEN_NL as a string, not a command
GITHUB_TOKEN_NL="cmd:gh auth token"
GITHUB_TOKEN="cmd:echo -n $GITHUB_TOKEN_NL"

For the mean time I just have to move this out of the .env file, which is inconvenient since it's the only token which needs to exist outside of the .env file.

Edit: one option I've found to work is parsing the secret within a call. Inconvenient, but it works

func stripSecret(ctx context.Context, s *dagger.Secret) (*dagger.Secret, error) {
	if s == nil {
		return nil, nil
	}
	uri, err := s.URI(ctx) // s.Name is empty, use URI instead
	if err != nil {
		return nil, err
	}
	plain, err := s.Plaintext(ctx)
	if err != nil {
		return nil, err
	}
	plain = strings.TrimSpace(plain)
	return dag.SetSecret(uri+"-strip", plain), nil
}