`fly secrets import`should take into account values between double quotes

[PierBover]

When using .env files, it's very common to add single line certificates like so:

MY_CERT="-----BEGIN CERTIFICATE-----\nMIIFaz... etc ...FfLn\n-----END CERTIFICATE-----\n"

When using fly secrets import these vars get converted to:

MY_CERT='"-----BEGIN CERTIFICATE-----\\nMIIFaz... etc ...FfLn\\n-----END CERTIFICATE-----\\n"'

Which of course breaks the application as new quotes are added, and line breaks are escaped.

It's definitely possible to add a secret from a cert file with:

flyctl secrets set MY_CERT=- < my-cert.crt

But there are cases when the key is already in a single line form (eg: Google Cloud service account).

It would be nice if import detected when a value is already between quotes, and simply saved the string as is without escaping or adding new quotes.

[amithm7]

I have used flyctl secrets import < .env too, but had to remove quotes around all variables. This is probably because redirection operator escapes special characters. And fly secrets import is not expecting that.

[jsierles]

Thanks for raising this. This and #575 are a good small task for someone!

[me-diru]

Hello, may I work on this? thanks!

[jsierles]

Feel free!

[me-diru]

Is this still the case? I was able to add the same certificate via flyctl secrets import < .env

the output:

$ ../flyctl/bin/flyctl secrets import < .env 

map[MY_CERT:"-----BEGIN CERTIFICATE-----\nMIIFaz... etc ...FfLn\n-----END CERTIFICATE-----\n"]
Release v2 created
Monitoring Deployment

v2 is being deployed
b9140e6b: maa pending
b9140e6b: maa pending
--> v2 deployed successfully

maybe I doing it wrong somewhere as I was unable to re-create the bug. also after doing this a couple of times, I got a Error Could not resolve VaultSecret error now

[PierBover]

Is there any other way to add a multi line cert into a secret?

Creating a secret with quotes and line break characters still doesn't work, and now I'm trying to do this:

fly secrets set SOME_KEY=- < my_file.cert

And I'm getting:

Error Post "https://api.fly.io/graphql": unexpected EOF

Edit:

Sorry, apparently the secrets API was down. It's working now when importing the file.

[PierBover]

This still seems broken.

I tried to import with fly secrets import < .env that had a single line cert with quotes and my app got this error:

error:0909006C:PEM routines:get_name:no start line

Which I'm assuming means the format of the cert is broken.

[NOBLES5E]

flyctl shouldn't add additional quotes. This breaks so many use cases.

[m4rvr]

So, is this fixed now or what's the workaround for it?

[HoaX7]

yeah okay, i've been stuck on this for hours and finally found this open issue,. is there a workaround? or should i just add my service account json into docker, which is not ideal.

[mboyea]

Hey all. While https://github.com/superfly/flyctl/pull/2476 seems to be complete, I still encountered this issue.

I am using the following workaround. I wrote a bash shell script to remove any quotes from each $input_line in the .env flyctl_env_vars+=$(echo "$input_line " | tr -d '"') The full script to set secrets is as follows:

#!/bin/bash

# gather env secrets for flyctl
flyctl_env_vars=("$@")
while IFS= read -r input_line || [[ -n "$input_line" ]]; do
	input_line=$(echo $input_line)
	# remove quotes from flyctl env (until https://github.com/superfly/flyctl/issues/589 is resolved)
	flyctl_env_vars+=$(echo "$input_line " | tr -d '"')
done < .env

# set flyctl env secrets
flyctl secrets set ${flyctl_env_vars[@]}

If you want to pass the secrets into Docker too when you deploy, consider using this script to deploy both at once:

#!/bin/bash

# gather env secrets for flyctl and docker
docker_env_vars=("$@")
flyctl_env_vars=("$@")
while IFS= read -r input_line || [[ -n "$input_line" ]]; do
	input_line=$(echo $input_line)
	docker_env_vars+="--build-secret $input_line "
	# remove quotes from flyctl env (until https://github.com/superfly/flyctl/issues/589 is resolved)
	flyctl_env_vars+=$(echo "$input_line " | tr -d '"')
done < .env

# set flyctl env secrets
flyctl secrets set --stage ${flyctl_env_vars[@]}

# deploy app with env secrets passed through flyctl to docker
flyctl deploy -a www-babalada-com ${docker_env_vars[@]}

[redjonzaci]

From what I understand, you want this to work with both single-line and multi-line certs?

[redjonzaci]

What works for me:

• fly secrets set MY_CERT="CERT\n"
• fly secrets set --stage MY_CERT="CERT\n"
• fly secrets import < .env

[benschenker]

fly secrets import < .env prints Oops, something went wrong! Could you try that again? When I have a multiline cert like:

PRIVATE_KEY="
-----BEGIN PRIVATE KEY-----
blahblah
-----END PRIVATE KEY-----
"

but it works perfectly fine without the multiline cert

[redjonzaci]

@benschenker thanks for the response! Checking it now.

[redjonzaci]

@benschenker would it work for you if you did:

PRIVATE_KEY="""
-----BEGIN PRIVATE KEY-----
blahblah
-----END PRIVATE KEY-----
"""

Because that's how multi-line is handled currently.

[benschenker]

Oh wow that seems to work! Can we find a way to document this well? I'd be happy to help provide some language for the https://fly.io/docs/flyctl/secrets-import/ page. Honestly just having a set of examples would be good enough.

[redjonzaci]

@jsierles where would we add the docs for this? Is the top of the page @benschenker mentioned good enough?

[redjonzaci]

@andie787 sorry to tag you here, but I have noticed you work on docs and maybe you could help us out with this?

[andie787]

@redjonzaci I can look at a doc update for the multiline syntax. It's not clear to me though if the original issue of adding quotes was addressed?

[redjonzaci]

What works for me:

• fly secrets set MY_CERT="CERT\n"

• fly secrets set --stage MY_CERT="CERT\n"

• fly secrets import < .env

@andie787 I tested these 10 days ago and they worked for me. These were mentioned as not working properly before I got here.

[redjonzaci]

Too many people to tag here, but if anyone is still having the original issue, we would appreciate a response.

[abegehr]

I'm running into this issue too. The secret is not double quoted but the new line \n are escaped to \\n which breaks my pem key.

Example:

fly secrets set MY_CERT="-----BEGIN CERTIFICATE-----\nMIIFaz... etc ...FfLn\n-----END CERTIFICATE-----\n"

becomes this when accessed on the machine:

-----BEGIN CERTIFICATE-----\\nMIIFaz... etc ...FfLn\\n-----END CERTIFICATE-----\\n

(notice \\n instead of \n)

When setting the key as multiline string """ and newlines instead of \n it works:

fly secrets import
MY_CERT="""-----BEGIN CERTIFICATE-----
MIIFaz... etc ...FfLn
-----END CERTIFICATE-----
"""

EDIT: Reference to recent related forum post: https://community.fly.io/t/how-to-store-pem-p8-key-in-secrets/15784

[jckw]

Also confirming \n gets converted to \\n. This is also a problem in the UI, since there's no way to enter multiline secrets. Solving this would solve both I think