taskcat

taskcat copied to clipboard

Describe the bug Taskcat not able to access S3 buckets

For Example: This is how the TemplateURL is given in CloudFormation Template: https://github.com/awslabs/aws-centralized-logging/blob/master/deployment/centralized-logging-primary.template

 TemplateURL: !Join 
        - ''
        - - 'https://'
          - !Join ["-", [!FindInMap ["SourceCode", "General", "S3Bucket"], "reference"]]
          - '.s3.amazonaws.com/'
          - !Join ["/", [!FindInMap ["SourceCode", "General", "KeyPrefix"],  "centralized-logging-demo.template"]]

Taskcat Error:

[WARN   ] : Failed to discover path for {'Fn::Join': ['', ['https://', {'Fn::Join': ['-', [{'Fn::FindInMap': ['SourceCode', 'General', 'S3Bucket']}, 'reference']]}, '.s3.amazonaws.com/', {'Fn::Join': ['/', [{'Fn::FindInMap': ['SourceCode', 'General', 'KeyPrefix']}, 'centralized-logging-demo.template']]}]]}, path {'Fn::Join': ['/', [{'Fn::FindInMap': ['SourceCode', 'General', 'KeyPrefix']}, 'centralized-logging-demo.template']]} does not exist
[ERROR  ] : Lint detected issues for test test1us on template /Users/dadmukta/Centralized_logging_taskcat/centralized-logging-primary.template:
[ERROR  ] :     line 338 [3030] [Check if properties have a valid value] You must specify a valid value for Runtime (nodejs12.x). Valid
                                                               values are ['dotnetcore1.0', 'dotnetcore2.0', 'dotnetcore2.1',
                                                               'go1.x', 'java8', 'nodejs', 'nodejs4.3-edge', 'nodejs4.3',
                                                               'nodejs6.10', 'nodejs8.10', 'nodejs10.x', 'provided', 'python2.7',
                                                               'python3.6', 'python3.7', 'ruby2.5']
[ERROR  ] :     line 447 [3012] [Check resource properties values] Property Resources/ElasticsearchAWSLogs/Properties/SnapshotOptions/Autom
                                                         atedSnapshotStartHour should be of type Integer
[ERROR  ] :     line 467 [3012] [Check resource properties values] Property Resources/ElasticsearchAWSLogs/Properties/AdvancedOptions/indic
                                                         es.fielddata.cache.size should be of type String
[ERROR  ] :     line 788 [3030] [Check if properties have a valid value] You must specify a valid value for Runtime (nodejs12.x). Valid
                                                               values are ['dotnetcore1.0', 'dotnetcore2.0', 'dotnetcore2.1',
                                                               'go1.x', 'java8', 'nodejs', 'nodejs4.3-edge', 'nodejs4.3',
                                                               'nodejs6.10', 'nodejs8.10', 'nodejs10.x', 'provided', 'python2.7',
                                                               'python3.6', 'python3.7', 'ruby2.5']
[ERROR  ] : Lint failed with errors
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/taskcat/_cli.py", line 70, in main
    cli.run()
  File "/usr/local/lib/python3.7/site-packages/taskcat/_cli_core.py", line 235, in run
    return getattr(command(), subcommand)(**args)
  File "/usr/local/lib/python3.7/site-packages/taskcat/_cli_modules/test.py", line 67, in run
    raise TaskCatException("Lint failed with errors")
taskcat.exceptions.TaskCatException: Lint failed with errors

To Reproduce Steps to reproduce the behavior:

1. Are you testing a QuickStart or Custom template? - Custom Template
2. Attach or link a copy of the template if possible (remove any sensitive info) - cannot share template
3. Provide the parameters that you passed. (remove any sensitive info) - Provided above
4. How did you install taskcat? (docker or pip3) - pip3
5. Are you using a profile, an instance role or access keys to run taskcat? - access keys
6. Is your AWS environment configured via aws configure? - yes

Expected behavior Taskcat should be able to access S3 files. We do not want to save all the files locally.

Screenshots If applicable, add screenshots to help explain your problem.

**Version (Please make sure you are running the latest version of taskcat)

• Taskcat Version (ex: [2018.817.210357])

Note: Python Version (python3 required)

To find versions: Via taskcat: taskcat -V Via pip3: pip3 show taskcat

Note: both version should match

To update taskcat run: for docker : docker pull taskcat/taskcat for pip3: pip3 install --upgrade taskcat

Additional context Add any other context about the problem here.

This looks like a duplicate (or very similar to) #446.

Confirmed duplicate (or varation of) #446; closing.

re-opening as the url pattern defined here is different to the pattern in #446 and we'll want to be sure the upcoming fix in #481 works for both.

/cc @gargana - FYI

Hello everyone! What about this issue? How can I solve this problem?

13:01:15 boto3.exceptions.S3UploadFailedError: Failed to upload /home/user/workspace/quickstart-amazon-eks-test/output/build/NOTICE.txt to mybucket/quickstart-amazon-eks-test-versions/NOTICE.txt: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
13:01:15 taskcat.exceptions.TaskCatException: Failed to upload to S3