Unable to access /app/.aws for configuring ECR credentials after 0.13.0 upgrade

[schlags]

Describe the bug It seems like this upgrade has (maybe?) broken functionality for using an authentication script for ECR like the following:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-image-updater-authscripts
data:
  # The following script must have a newline at the end of the output to work correctly, so we explicitly echo the output
  # https://argocd-image-updater.readthedocs.io/en/stable/basics/authentication/#using-a-script-to-generate-credentials
  ecr-login.sh: |
    #!/bin/sh

    userpass=$(aws ecr --region {replace with your region} get-authorization-token --output text --query 'authorizationData[].authorizationToken' | base64 -d) && echo $userpass

To Reproduce Steps to reproduce the behavior:

1. Upgrade to v0.13.0
2. Define the above snippet for the argocd-image-updater-authscripts ConfigMap.
3. Set the annotations on an application for a private ECR registry
4. See logs in the image updater service along the lines of

   Could not set registry endpoint credentials: invalid script output, must be single line with syntax <username>:<password>
   

5. Shell into the pod and run the command directly and see the following output:

   
   / $ /scripts/ecr-login.sh
   
   [Errno 30] Read-only file system: '/app/.aws'
   

Expected behavior A clear and concise description of what you expected to happen.

Like the behavior in v0.12.2 this should be acceptable to run aws commands on the pod. Running this command locally does indeed output a single line that follows the username:password convention, but the issue is with the new securityContext definition on the container spec.

AWS:authorization-token

Additional context Add any other context about the problem here.

Reverting back to v0.12.2 immediately fixed the issue, and logs do not return.

Version Please tell us about the version you encountered the issue with

v0.13.0

Logs Please paste any relevant logs here

time="yyyy-MM-ddTHH:mm:ssXXX" level=error msg="Could not set registry endpoint credentials: invalid script output, must be single line with syntax <username>:<password>" alias=* application=* image_name=*/* image_tag=* registry=REDACTED.dkr.ecr.us-west-2.amazonaws.com

[schlags]

/ $ /scripts/ecr-login.sh

[Errno 30] Read-only file system: '/app/.aws'

It looks like this is the issue. Regardless of how the script gets mounted, it looks like using the aws cli is not possible with the current default securityContext

[schlags]

Removing the securityContext block seems to resolve this issue. Let me know if there is a better way to handle this aside from deleting that block.

[chapayevdauren]

the same here

[chengfang]

@schlags @chapayevdauren thanks for reporting the issue. Are you referring to the following block, which was added in PR #600 ? https://github.com/argoproj-labs/argocd-image-updater/blob/master/manifests/install.yaml#L189

Will it work if you change the file system attribute to readOnlyRootFilesystem: false?

[jannfis]

You could probably also just configure the aws CLI to read/write configuration in a different location by setting the AWS_CONFIG_FILE environment to point to a read-write filesystem in the container, for example:

AWS_CONFIG_FILE=/tmp/.aws/config

[jannfis]

To properly fix this, we might want to have an emptyDir mount for /app/.aws in the installation manifests.

[schlags]

Will it work if you change the file system attribute to readOnlyRootFilesystem: false?

I tested multiple configurations of the block initially (with this included) and had no luck.

[choge]

FYI, I had the same issue and worked around it by adding HOME=/tmp before the command. I think this is because AWS CLI tries to write session information to ~/.aws/cli/cache.

HOME=/tmp aws ecr --region $AWS_REGION get-authorization-token --output text --query 'authorizationData[].authorizationToken' | base64 -d

[jjnunogarcia]

FYI, I had the same issue and worked around it by adding HOME=/tmp before the command. I think this is because AWS CLI tries to write session information to ~/.aws/cli/cache.

HOME=/tmp aws ecr --region $AWS_REGION get-authorization-token --output text --query 'authorizationData[].authorizationToken' | base64 -d

I can confirm this works also when using gcloud.

[phyzical]

i stumbled across this dealing with the same issue in a totally unrelated context

FYI @jannfis using a emptyDir solved it for my usecase perfectly

So i would recommend it for a long term solution in this project

[antonioned]

This is still an ongoing issue even in v0.15.1. I can confirm adding HOME=/tmp fixes it but I agree with everyone that a definitive solution for this is needed.