Driftctl sometimes incorrectly calculating aws_route IDs

[dankeder]

Description

Drifctl is sometimes reporting false positives with resource aws_route (provider hashicorp/aws).

Environment

• OS: Fedora 36
• driftctl version: v0.35.2
• terraform version: Terraform v1.2.5 on linux_amd64
• terraform providers versions: hashicorp/[email protected]

How to reproduce

Let's say we have a route table rtb-XXXXXXXXXXXXXXXXX which contains a route for destination 172.18.0.0/22 in AWS. The route table is already present in terraform state:

$ terraform state show 'aws_route.my_little_route'

prints:

# aws_route.my_little_route:
resource "aws_route" "my_little_route" {
    destination_cidr_block    = "172.18.0.0/22"
    id                        = "r-rtb-XXXXXXXXXXXXXXXXX4165916864"
    origin                    = "CreateRoute"
    route_table_id            = "rtb-XXXXXXXXXXXXXXXXX"
    state                     = "active"
    vpc_peering_connection_id = "pcx-YYYYYYYYYYYYYYYYY"
}

Drifctl shows this route as not present in Terraform state:

driftctl scan --only-unmanaged --from "tfstate+s3://terraform-state/**/*.tfstate" --filter "Type == 'aws_route'" --quiet

prints the resource as not covered by IaC:

  aws_route:
    - r-rtb-XXXXXXXXXXXXXXXXX129050432
        Destination: 172.18.0.0/22, Table: rtb-XXXXXXXXXXXXXXXXX

Note the difference in route IDs calculated by terraform and driftctl: r-rtb-XXXXXXXXXXXXXXXXX4165916864 vs r-rtb-XXXXXXXXXXXXXXXXX129050432. As far as I can tell these IDs are calculated as r-{route_table_id}{crc32_of_destination}.

After investigating this further it looks like driftctl is performing signed int CRC32 while terraform unsigned int CRC32. Check out this Python snippet:

>>> binascii.crc32(b'172.18.0.0/22')             # this is what terraform works with
4165916864
>>> -binascii.crc32(b'172.18.0.0/22') % (1<<32)  # this is what driftctl works with
129050432

[eliecharra]

Thanks a lot for that super complete bug report @dankeder 🙏🏻 This little route is super cute 😆

Here I suspect that the way of computing the ID has changed in latest provider versions (yeah I'm lazy I've not checked the provider changelog) This is probably a major BC break introduced in aws provider 4.x, we probably have a lot of things to handle on our side.

Note for maintainers, we need to introduce some versioned behavior and assert on the provider version currently in use. We could retrieve that from the resource.Resource.Schema().SchemaVersion or resource.Resource.Schema().ProviderVersion. Please use schema version if possible, but I'm not sure how they handle the bump of the version on the provider side, not sure if a change in ID computation introduce a schema breaking change from their perspective 🤔

[sundowndev-snyk]

Hi @dankeder, I've checked the provider's changelog and couldn't find any information about such change. Unfortunately I didn't manage to reproduce the bug with the details you provided. Could you please provide a HCL snippet that we can use to reproduce this? Also I noticed you didn't use the flag --tf-provider-version, which can lead to inconsistencies. By default, the tool uses [email protected] which is outdated in your case.

[craigfurman]

@eliecharra this doesn't look (at first glance) like a BC break around how the routeID is calculated. Both 3.X and main (4.X currently) of the provider use the same code to calculate CRCs and cast them to non-negative ints ("int31"-ish): https://github.com/hashicorp/terraform-provider-aws/blob/39de967ca0e41f7848dd26441355ff6f162db0b2/aws/internal/hashcode/hashcode.go and https://github.com/hashicorp/terraform-provider-aws/blob/main/internal/create/hashcode.go, called from https://github.com/hashicorp/terraform-provider-aws/blob/2dbb620bea4735173c8a84077f59e586b4ef8f35/internal/service/ec2/id.go#L30 and https://github.com/hashicorp/terraform-provider-aws/blob/39de967ca0e41f7848dd26441355ff6f162db0b2/aws/resource_aws_route.go#L543 respectively. I checked provider version 2.X too, the exact same impl.

When enumerating resources, driftctl uses an old-ish version of the terraform SDK (https://github.com/snyk/driftctl/blob/main/enumeration/resource/aws/aws_route.go), and the implementation of the hashcode is the same as always 🤔

@dankeder, thanks for pointing us in the direction of the checksum calculation here. Your IDs indeed look like they come from a discrepancy in hashcode overflow/casting implementations, but we are struggling to pinpoint where that discrepancy might be arising. I think we need a few more pieces of information:

• In a driftctl scan that reproduces this bug, can you let us know what provider version is printed after "Provider version used to scan:"?
• You mentioned that you are using the AWS provider version 4.19.0. Am I right to assume that this is the value present in your .terraform.lock.hcl, if one is present?
• Am I also right to assume that your terraform state was most recently written by provider version 4.19.0? If it hasn't been written since a provider upgrade, do you know which provider version most recently wrote the state?

[craigfurman]

Since we couldn't reproduce this and the thread seems to have gone quiet, I'll close this. Please feel free to reopen if this is still an issue (and please see the above comment asking for more information).