pulumi-aws icon indicating copy to clipboard operation
pulumi-aws copied to clipboard

Consider running upstream tests

Open t0yv0 opened this issue 2 years ago • 1 comments

Consider enabling running upstream tests on our patched version of the provider as part of CI verification.

Currently these tests do not compile with our patches:

$ make upstream && cd upstream && go test ./... -test.run IGNORE
internal/service/s3legacy/bucket_legacy.go:22:2: no required module provides package github.com/hashicorp/aws-sdk-go-base/tfawserr; to add it:
        go get github.com/hashicorp/aws-sdk-go-base/tfawserr

It is possible that some of them would need to be disabled as they're not compatible with Pulumi patches.

t0yv0 avatar Nov 17 '23 19:11 t0yv0

Our first attempt to run the tests exposed some inadequate support in the testing infra to be able to support this atm (see https://github.com/pulumi/home/issues/3599).

We decided to pivot to testing selectively the services that need it most because of patches in pulumi-aws and the risk of these patches introducing issues not detected by upstream testing.

Here is a table of services to target:

| s3legacy        | 0002-Add-S3-legacy-bucket-to-resources.patch                     |
| s3              |                                                                  |
| lakeformation   | 0005-Remove-lakeformation-catalog_resource-default.patch         |
| ssm             | 0006-Workaround-SSM-Parameter-tier-bug.patch                     |
| eks             | certificate_authorities? Add-ESK-cluster-certificate-authorities |
| autoscaling     | 0008-Workaround-Autoscaling-launch_configuration-associat.patch  |
| ecr             | Add ECR credentials_data_source                                  |
| lambda          | 0054-Parallelize-Lambda-Function-resource-operations.patch       |
|                 | 0055-Create-Logging-Middleware-for-Lambda-service-that-do.patch  |
|                 | 0035-Disable-retry-for-KMS-access-denied-in-lambda.patch         |
| acm             | 0036-Patch-ACM-retry-to-not-retry-after-LimitExceededExce.patch  |
| sns             | 0048-non-idempotent-sns-topic-creation.patch                     |
|                 | 0013-Catch-cty-panic-in-new-resourceTopicSubscriptionCust.patch  |
| controltower    | 0049-Normalize-retentionDays-in-aws_controltower_landing_.patch  |
| rds             | 0050-Revert-r-aws_db_proxy-Change-auth-from-TypeList-to-T.patch  |
| ds              | 0051-Fix-panic-in-aws_directory_service_directory-Read.patch     |
| elbv2           | 0021-Fix-elbv2-target-group-read-to-workaround-2517.patch        |
| sqs             | 0022-Fix-spurrious-json-diff-for-redrive_policy.patch            |
| eks             | 0028-aws_eks_cluster-implement-default_addons_to_remove.patch    |
| waf             | 0012-Revert-WAF-schema-changes.patch                             |
| gamelift        | 0064-Adapt-gamelift-matchmaking-resources.patch                  |
| route53resolver | 0016-Revert-Update-endpointHashIPAddress.patch                   |

t0yv0 avatar Aug 21 '24 21:08 t0yv0