EC2 Transit Gateway attachment associations cannot be non-default

[Zordrak]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Core: 0.11.10
AWS Provider: 1.50.0

Affected Resource(s)

• aws_ec2_transit_gateway
• aws_ec2_transit_gateway_vpc_attachment
• aws_ec2_transit_gateway_route_table
• aws_ec2_transit_gateway_route_table_association
• aws_ec2_transit_gateway_route_table_propagation

Terraform Configuration Files

resource "aws_ec2_transit_gateway" "main" {
  description                     = "${local.csi}"
  default_route_table_association = "disable"
  default_route_table_propagation = "disable"
}

resource "aws_ec2_transit_gateway_vpc_attachment" "ctrl" {
  transit_gateway_id = "${aws_ec2_transit_gateway.main.id}"
  vpc_id             = "${aws_vpc.main.id}"

  transit_gateway_default_route_table_association = "false"
  transit_gateway_default_route_table_propagation = "false"

  subnet_ids = [
    "${module.subnets_tgw_ctrl.subnet_ids}",
  ]
}

resource "aws_ec2_transit_gateway_route_table" "ctrl" {
  transit_gateway_id = "${aws_ec2_transit_gateway.main.id}"
}

resource "aws_ec2_transit_gateway_route_table_association" "ctrl_ctrl" {
  transit_gateway_attachment_id  = "${aws_ec2_transit_gateway_vpc_attachment.ctrl.id}"
  transit_gateway_route_table_id = "${aws_ec2_transit_gateway_route_table.ctrl.id}"
}

Debug Output

I know you don't like debug pastes.. but it's short enough.

---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: ec2.eu-west-1.amazonaws.com
User-Agent: aws-sdk-go/1.15.84 (go1.11.1; linux; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.11.9-beta1
Content-Length: 183
Authorization: AWS4-HMAC-SHA256 Credential=<REDACTED>/20181130/eu-west-1/ec2/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=<REDACTED>
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20181130T161403Z
Accept-Encoding: gzip

Action=GetTransitGatewayRouteTableAssociations&Filter.1.Name=transit-gateway-attachment-id&Filter.1.Value.1=tgw-attach-<REDACTED>&TransitGatewayRouteTableId=&Version=2016-11-15
-----------------------------------------------------
2018/11/30 16:14:03 [DEBUG] [aws-sdk-go] DEBUG: Response ec2/GetTransitGatewayRouteTableAssociations Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 400 Bad Request
Connection: close
Transfer-Encoding: chunked
Date: Fri, 30 Nov 2018 16:13:50 GMT
Server: AmazonEC2


-----------------------------------------------------
2018/11/30 16:14:03 [DEBUG] [aws-sdk-go] <?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>MissingParameter</Code><Message>Missing required parameter in request: TransitGatewayRouteTableId.</Message></Error></Errors><RequestID><REDACTED></RequestID></Response>
2018/11/30 16:14:03 [DEBUG] [aws-sdk-go] DEBUG: Validate Response ec2/GetTransitGatewayRouteTableAssociations failed, not retrying, error MissingParameter: Missing required parameter in request: TransitGatewayRouteTableId.
     status code: 400, request id: <REDACTED>

* aws_ec2_transit_gateway_vpc_attachment.ctrl: aws_ec2_transit_gateway_vpc_attachment.ctrl: error determining EC2 Transit Gateway Attachment (tgw-attach-<REDACTED>) association to Route Table (): MissingParameter: Missing required parameter in request: TransitGatewayRouteTableId.
        status code: 400, request id: <REDACTED>

Expected Behavior

EC2 Transit Gateway VPC Attachment should be created without error.

Actual Behavior

If either a Transit Gateway Route Table Association or Transit Gateway Route Table Propagation is not defined for the EC2 Transit Gateway VPC Attachment, after the Attachment resource is created, terraform attempts to confirm creation success by describing the Attachment resource and its properties. This describe step fails because terraform tries to describe a route table association and or propagation that does not exist and then exits on failure; despite neither of these properties being required.

Despite the exit, the Attachment resource remains successfully created as the Create API call succeeded, only the subsequent describe fails.

Steps to Reproduce

1. terraform apply

Important Factoids

If you work around this issue by allowing the EC2 Transit Gateway to associate VPC Attachments with a default route table and/or propagation, then the attachment can be successfully created and terraform can successfully describe the Attachment resource and move on.

However if you do this you cannot then use aws_ec2_transit_gateway_route_table_association and aws_ec2_transit_gateway_route_table_propagation resources because these things are already defined for the Attachment and the API call to associate them does not allow for automatic replacement.

As a result, these resources currently cannot be used as either defaults are used, and so they cannot be used; or the attachment creation fails, and terraform exits before they can be applied.

References

#6605

https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_GetTransitGatewayRouteTableAssociations.html

https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_GetTransitGatewayRouteTablePropagations.html

https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeTransitGatewayVpcAttachments.html

https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateTransitGatewayVpcAttachment.html

https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_AssociateTransitGatewayRouteTable.html

[bflad]

The initial acceptance testing was written with only one of the two flags disabled at a time. I'm able to reproduce the error when both are disabled:

--- FAIL: TestAccAWSEc2TransitGatewayVpcAttachment_TransitGatewayDefaultRouteTableAssociationAndPropagationDisabled (87.32s)
    testing.go:538: Step 0 error: Error applying: 1 error occurred:
        	* aws_ec2_transit_gateway_vpc_attachment.test: 1 error occurred:
        	* aws_ec2_transit_gateway_vpc_attachment.test: error updating EC2 Transit Gateway Attachment (tgw-attach-0e5e1795143bd2b2e) Route Table () association: error determining EC2 Transit Gateway Attachment Route Table () association (tgw-attach-0e5e1795143bd2b2e): MissingParameter: Missing required parameter in request: TransitGatewayRouteTableId.

I'll work on a fix shortly.

[bflad]

Fix pull request submitted: #6665

A workaround for this specific error until the above fix is released is to leave one of the aws_ec2_transit_gateway attributes for default route table association/propagation set to true so the TGW default route table exists, which ensures the VPC attachment resource has a route table to search against.

[amall015]

I experienced a similar issue where I am creating an aws_ec2_transit_gateway_vpc_attachment using an aws_ec2_transit_gateway that was created in another account and shared with this one using the AWS Resource Access Manager. Because the TGW is shared, the default route table does not appear in the context of the shared account even though it can be seen through the TGW resource itself, so terraform throws a similar error to what is described above (pasted below). I thought that by setting both transit_gateway_default_route_table_association and transit_gateway_default_route_table_propagation to false, it would skip this check, but terraform still seems to be trying looking up the default route table. To note, the aws_ec2_transit_gateway_vpc_attachment is created successfully, but the result of the run and all subsequent plans fail with the error below.
module.network.module.transit_gateway.aws_ec2_transit_gateway_vpc_attachment.VPC-Attachment: aws_ec2_transit_gateway_vpc_attachment.VPC-Attachment: error determining EC2 Transit Gateway Attachment (tgw-attach-#######) association to Route Table (tgw-rtb-#######): InvalidRouteTableID.NotFound: Transit Gateway Route Table tgw-rtb-###### was deleted or does not exist.

[bflad]

@amall015 thanks for the heads up. Could you please create a separate issue for that one? Multi account setup was not available during the beta so there are likely some additional rough edges with that support as it’s not tested as well. I’d imagine we might introduce a fix there to ignore that error.

[amall015]

@bflad Thanks for looking into it. Created issue #6670 for this.

[bflad]

The fix for the original issue has been merged and will release with version 1.51.0 of the AWS provider, likely in the next day or two. 👍

[bflad]

This has been released in version 1.51.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[piersf]

@bflad i'm still getting the first original issue. I'm running Terraform v0.11.11 with AWS Provider version 1.54.0

* aws_ec2_transit_gateway_vpc_attachment.transit_gateway_vpc_attachment: error updating EC2 Transit Gateway Attachment (tgw-attach-xxxxxxxxxxxx) Route Table () association: error associating EC2 Transit Gateway Route Table () association (tgw-attach-xxxxxxxxxxxxxxxx): MissingParameter: Missing required parameter in request: TransitGatewayRouteTableId.7

[piersf]

Tried also with AWS Provider version 1.56.0 and I still see the same error.

Are we sure this has been fixed? Can somebody confirm please?

[piersf]

Tried also with AWS Provider version 1.51.0 specifically in case this fix somehow did not get in the later versions and the error remains.

[piersf]

For anyone else encountering this, see the complete solution in #7220.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!