please icon indicating copy to clipboard operation
please copied to clipboard

Environment variables not available for custom cache command

Open adox opened this issue 2 months ago • 1 comments

AWS S3 custom cache fails (configuration taken from the docs):

[Cache]
RetrieveCommand="aws s3 cp s3://YOUR-OWN-CACHE-BUCKET/please/$CACHE_KEY -"
StoreCommand="aws s3 cp - s3://YOUR-OWN-CACHE-BUCKET/please/$CACHE_KEY"

It looks like no external environment variables are available during the cache phase (ie AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) and authorization fails. Adding them with PassEnv / PassUnsafeEnv doesn't help.

Workaround was to store secrets in a file and modify the command to:

RetrieveCommand="source aws.sh && aws s3 cp s3://YOUR-OWN-CACHE-BUCKET/please/$CACHE_KEY -"

but this is ugly. I guess, PassEnv / PassUnsafeEnv should be respected during the cache phase or pharaphs all host environment variables should be made available.

adox avatar Oct 25 '25 17:10 adox

This might be a very simple fix: https://github.com/thought-machine/please/pull/3439

adox avatar Oct 26 '25 06:10 adox