Error using data.aws_vpc_endpoint_service in AWS US GovCloud

[lorengordon]

Terraform Version

> terraform -v
Terraform v0.11.7
+ provider.aws v1.16.0

Affected Resource(s)

Please list the resources as a list, for example:

• data.aws_vpc_endpoint_service

Terraform Configuration Files

Straight from the example:

# Declare the data source
data "aws_vpc_endpoint_service" "s3" {
  service = "s3"
}

# Create a VPC
resource "aws_vpc" "foo" {
  cidr_block = "10.0.0.0/16"
}

# Create a VPC endpoint
resource "aws_vpc_endpoint" "ep" {
  vpc_id       = "${aws_vpc.foo.id}"
  service_name = "${data.aws_vpc_endpoint_service.s3.service_name}"
}

Debug Output

https://gist.github.com/lorengordon/f10b04a991edd0f1474bacc9ea1a3146

Expected Behavior

The data.aws_vpc_endpoint_service should return the service name just as it does for the commercial regions.

Actual Behavior

Got the error, multiple VPC Endpoint Services matched; use additional constraints to reduce matches to a single VPC Endpoint Service.

Steps to Reproduce

1. terraform plan

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

data.aws_vpc_endpoint_service.s3: Refreshing state...

Error: Error refreshing state: 1 error(s) occurred:

* data.aws_vpc_endpoint_service.s3: 1 error(s) occurred:

* data.aws_vpc_endpoint_service.s3: data.aws_vpc_endpoint_service.s3: multiple VPC Endpoint Services matched; use additional constraints to reduce matches to a single VPC Endpoint Service

[ewbankkit]

It looks like the functionality to filter VPC endpoint services by service name isn't working in GovCloud:

2018-04-30T10:52:40.782-0400 [DEBUG] plugin.terraform-provider-aws_v1.16.0_x4.exe: 2018/04/30 10:52:40 [DEBUG] Reading VPC Endpoint Services: {
2018-04-30T10:52:40.782-0400 [DEBUG] plugin.terraform-provider-aws_v1.16.0_x4.exe:   ServiceNames: ["com.amazonaws.us-gov-west-1.s3"]
2018-04-30T10:52:40.782-0400 [DEBUG] plugin.terraform-provider-aws_v1.16.0_x4.exe: }
...
2018-04-30T10:52:41.305-0400 [DEBUG] plugin.terraform-provider-aws_v1.16.0_x4.exe: 2018/04/30 10:52:41 [DEBUG] [aws-sdk-go] <?xml version="1.0" encoding="UTF-8"?>
2018-04-30T10:52:41.305-0400 [DEBUG] plugin.terraform-provider-aws_v1.16.0_x4.exe: <DescribeVpcEndpointServicesResponse xmlns="http://ec2.amazonaws.com/doc/2016-11-15/">
2018-04-30T10:52:41.305-0400 [DEBUG] plugin.terraform-provider-aws_v1.16.0_x4.exe:     <requestId><REQUESTID></requestId>
2018-04-30T10:52:41.305-0400 [DEBUG] plugin.terraform-provider-aws_v1.16.0_x4.exe:     <serviceNameSet>
2018-04-30T10:52:41.305-0400 [DEBUG] plugin.terraform-provider-aws_v1.16.0_x4.exe:         <item>com.amazonaws.us-gov-west-1.dynamodb</item>
2018-04-30T10:52:41.305-0400 [DEBUG] plugin.terraform-provider-aws_v1.16.0_x4.exe:         <item>com.amazonaws.us-gov-west-1.s3</item>
2018-04-30T10:52:41.305-0400 [DEBUG] plugin.terraform-provider-aws_v1.16.0_x4.exe:     </serviceNameSet>
2018-04-30T10:52:41.305-0400 [DEBUG] plugin.terraform-provider-aws_v1.16.0_x4.exe: </DescribeVpcEndpointServicesResponse>

[jmcarp]

If GovCloud doesn't support filtering on service name, we might want to revert to something like the logic we had in 1.8 when the region is GovCloud, or possibly if the number of results is greater than one:

https://github.com/terraform-providers/terraform-provider-aws/blob/8937a3a4e9d77c8089cf147861b604e3a2d8cf7e/aws/data_source_aws_vpc_endpoint_service.go#L44-L51

By the way, the acceptance tests for this data source hard-code regions and availability zones, so I can't run them in GovCloud. GovCloud also apparently only has endpoint services for s3 and dynamo, so tests for the ec2 endpoint service wouldn't work anyway. Any thoughts on making these tests more flexible about regions and skipping tests based on partition?

[lorengordon]

This was working under v1.12.0 when that was first released, but now even that version fails. So I think something about the response has changed since then, violating some assumption in the provider code.

[ewbankkit]

@lorengordon It could be that you have an additional VPC endpoint since the last successful time you ran - If the S3 endpoint was the only endpoint then the code would have worked.

@jmcarp Agree on the solution - The issue will be with testing; There was some talk about easing GovCloud testing but I'm not sure what the status on that is.

[lorengordon]

@ewbankkit I checked for multiple endpoints before reporting, because the error message appears to indicate something like that. But no, there is only a single endpoint in the account.

(It would actually be quite odd for there to ever be multiple endpoints, since I use this account solely for testing my terraform modules/workflows and nothing else is ever going on in the account. 🤷‍♂️ )

[acdifran]

I just discovered this same problem as well. To get around it I've been using aws provider v1.8.

[coffebene]

The same problem in China region~

[ewbankkit]

@Coffee-Bene #4592 should fix the cn- region issues.

[coffebene]

@ewbankkit Today,I test in my environment,it is ok ,thanks~

[bflad]

Sorry for the delay getting the fix merged -- it will release shortly with version 1.29.0 of the AWS provider.

[bflad]

This has been released in version 1.29.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[lorengordon]

Thanks @bflad and @ewbankkit!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!