resource/aws_ec2_transit_gateway_route: Crash from externally deleted attachment

[mskucherawy]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.10

• provider.aws v1.54.0
• provider.template v1.0.0

Affected Resource(s)

We suspect one or more of these based on the debug trace:

resource "aws_ec2_transit_gateway"
resource "aws_ec2_transit_gateway_vpc_attachment"
resource "aws_customer_gateway"
resource "aws_vpn_connection"
resource "aws_ec2_transit_gateway_route_table"
resource "aws_ec2_transit_gateway_route_table_association"
resource "aws_ec2_transit_gateway_route"

Terraform Configuration Files

These are possibly proprietary. If after reviewing the debug output you do need these, please let me know and I'll get them to you somehow.

Debug Output

https://gist.github.com/mskucherawy/f06c1a4ad7a945a08b7d81b2acb0f4a3

Panic Output

No "crash.log" was produced. See debug output.

Expected Behavior

This should complete, or produce a meaningful error message if there's not actually a bug.

Actual Behavior

panic; see debug log.

Steps to Reproduce

terraform plan

Important Factoids

We're using the Transit Gateway stuff, which is new.

References

None (yet).

[mskucherawy]

One possible contributor is that the customer gateway was previously applied, but then manually deleted via the console before this started happening.

[bflad]

Hey @mskucherawy 👋 Thanks for reporting this and sorry for the unexpected behavior here. Your additional context is super helpful! It looks like there's a typo in the resource code here:

https://github.com/terraform-providers/terraform-provider-aws/blob/70c134658608f6d56230adae0512aa73e28ab939/aws/resource_aws_ec2_transit_gateway_route.go#L127

That logic should be checking len(transitGatewayRoute.TransitGatewayAttachments) > 0 && transitGatewayRoute.TransitGatewayAttachments[0] != nil instead. Theoretically we should be able to write up a quick fix with a covering acceptance test (that creates the route then removes the attachment) before our next release. 👍

[mskucherawy]

Terrific!

What's the timeline for that release?

[bflad]

Pull request submitted: #7117

We have been releasing on a weekly cadence with an exception due to the year end holidays.

[mskucherawy]

Until this ships, can you suggest a workaround? For example, do I have to delete or taint the running transit gateway resources via the console to get "plan" and "apply" working again?

[bflad]

@mskucherawy you can use terraform state rm to remove troublesome resource from the Terraform state (it'll prompt to recreate next plan/apply) or use the -target flag with terraform plan/terraform apply to workaround the troublesome resource.

EDIT: Sorry you'll still have the same problem even after state rm and import so -target is your best bet until the fix is released or you can custom build the provider with the fix.

[mskucherawy]

Thanks! Are there docs online anyplace about building custom local providers?

[bflad]

This has been released in version 1.56.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!