SAM deploy complains about the config file, when the template file is not in current directory

[thogr]

Description:

SAM deploy complains about the config file (the --config-file) missing or unreadable, but it's really the about the path to the template file (the -t argument).  If the specified template file is not in the current directory the error from SAM is  Error: Configfile sam-config-file.toml does not exist or could not be read!

Steps to reproduce:

This gives the error: > sam deploy -t ./folder/my_template.yaml --config-file sam-config-file.toml --s3-bucket thebucket --region us-east-1 It doesn't matter if the path to the template file is absolute or relative. The template file must be in the current directory, like this: > sam deploy -t my_template.yaml --config-file sam-config-file.toml --s3-bucket thebucket --region us-east-1

Observed result:

2023-04-13 10:39:48,139 | Using SAM Template at /Users/tgronwall/sam-bugreport/folder/template.yaml 2023-04-13 10:39:48,139 | Config file sam-config-file.toml does not exist or could not be read! Error: Config file sam-config-file.toml does not exist or could not be read!

Expected result:

It should find the template file when I use a correct path (either absolute or relative) to a a template file in a sub folder, not only a file in the current directory. It should not complain about the config file, if it can't find the template file

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

Both MacOS and Linux (in our gitlab runner) I have tracked this bug to have been introduced in SAM v1.56. It did not occur in previous versions. See https://github.com/aws/aws-sam-cli/issues/3899

1. OS: Macos 12.6.4
2. sam --version: 1.78
3. AWS region: us-east-1

# Paste the output of `sam --info` here

{ "version": "1.78.0", "system": { "python": "3.8.16", "os": "macOS-12.6.4-x86_64-i386-64bit" }, "additional_dependencies": { "docker_engine": "20.10.12", "aws_cdk": "Not available", "terraform": "Not available" } }

Add --debug flag to command you are running

[thogr]

Same problem in v1.79: ➜ sam deploy --debug -t ./folder/template.yaml --config-file sam-config-file.toml --s3-bucket thebucket --region us-east-1 2023-04-13 12:56:22,949 | Using SAM Template at /Users/tgronwall/sam-bugreport/folder/template.yaml 2023-04-13 12:56:22,950 | Config file sam-config-file.toml does not exist or could not be read! Error: Config file sam-config-file.toml does not exist or could not be read! ➜ sam --info
{ "version": "1.79.0", "system": { "python": "3.8.16", "os": "macOS-12.6.4-x86_64-i386-64bit" }, "additional_dependencies": { "docker_engine": "20.10.12", "aws_cdk": "Not available", "terraform": "Not available" } }

[thogr]

Work-around is not using --config-file argument, but using the default config file name samconfig.toml.

[jfuss]

@thogr What does the file structure look like? Are you sure sam-config-file.toml exists in the directory you are running the command in? Looking at the code: https://github.com/aws/aws-sam-cli/blob/develop/samcli/cli/cli_config_file.py#L53-L56, we attempt to find the file based on the input and cwd.

[thogr]

@jruss Yes, I'm 100% sure it exists. In the real set up we actually have two files, one called sam-config-api.toml and one sam-config-db.toml. We call SAM with --config-file sam-config-api.toml and then call SAM with --config-file sam-config-db.toml. It worked perfectly before v1.56. The problem started when I tried to upgrade SAM to the latest version. The only thing changed was the version of SAM. To be able to upgraded SAM to the latest version, we need to first do a

cp sam-config-api.toml samconfig.toml

and then call sam without --config-file (since samconfig.toml is the default name) (and of-course the same with the sam-config-db.toml file)

[jfuss]

@thogr Just checking is all :)

I have a project based on sam init and able to have the right samconfig.toml be picked up. After init, I ran cp samconfig.toml samconfig-test.toml && rm samconfig.toml leaving me only with samconfig-test.toml. Then running sam validate --config-file samconfig-test.toml from the project directory, works as expected. Without the --config-file, I get a different output, which is expected since the toml enabled --lint on the sam validate command.

So this still leaves me puzzled. Can you described the project setup? Where are you running the SAM CLI command from and general project layout? I understand you have different files, but it doesn't help me understand the folder/file layout of your project. Which I assume is different from sam init's output.

[thogr]

Thanks @jfuss I'lll try to guide you to reproducing this. I have done a sam init. After that I did these steps, to somewhat mimic our file structure:

➜  mkdir folder
➜  mv template.yaml folder 
➜  mv samconfig.toml my_samconfig.toml
➜  sam deploy -t folder/template.yaml --config-file my_samconfig.toml
Error: Config file my_samconfig.toml does not exist or could not be read!

Same error for sam validate. Hopefully, you should be able reproduce it now.

I don't know if it matters, but as for the sam init, I chose these options: 1 - AWS Quick Start Templates 1 - Hello World Example 13 - nodejs16.x 2 - Image X-Ray: N CloudWatch Application Insights: N

[jfuss]

@thogr Moving the template to a new directory was the missing piece. If you run sam deploy -t folder/template.yaml --config-file ../my_samconfig.toml does it work as you expected? (It does on my machine but want to double check).

I would need to check, but what it looks like is we resolve the config-file to cwd if -t is not provided but if -t is provided, we solve the samconfig from that location. Sounds like what we may need to introduce then is "if --config-file is not found based on template, try based on cwd and if still not found error". Need to validate all of this, but just want it appears based on this thread

[thogr]

Thanks, @jfuss. Yes, the template file in a subdirectory is indeed of significance, as stated above in the bug report.

Yes, I can confirm that a work-around for this bug is to reference the config-file relative to the folder where the template file is. Works for me too. Thanks

[jfuss]

@thogr

Thanks, @jfuss. Yes, the template file in a subdirectory is indeed of significance, as stated above in the bug report.

I completely missed this in the initially part. Thanks for walking me through it and being patient while I was trying to understand what was happening.

[lucashuy]

Thanks for confirming this behaviour with us. We'll have to discuss this to see what our next steps are, whether this is something we make more clear in the documentation, something we can improve by considering the cwd of the command, or both.

[nascit]

Spent 1 hour questioning my sanity before finidng this issue +1

[lancerinf]

Same, spent a whole lot of time troubleshooting why:

• --template garmin-importer/template.yaml would correctly find the template.yaml while
• --config-file garmin-importer/samconfig.yaml would tell me that the file didn't exist

Indeed referencing the samconfig like --config-file samconfig.yaml worked.. but it's SO unintuitive that it should work like that :sob: .

[LearnerSamuelX]

spending half of my day troubleshooting this. the behaviour still exists, and no work-around approach (e.g having samconfig.toml removed from current directory) given in the documentation