-
Notifications
You must be signed in to change notification settings - Fork 9.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tests/provider: Standardize remaining alternate provider testing and refactor testAccProviderFactories for broad usage #15904
Conversation
…refactor testAccProviderFactories for broad usage Reference: #8316 Reference: #15737 Reference: #15791 This change set continues the work of #15877, bringing a consistent pattern for cross-region and cross-account testing and removing previously deprecated provider-level test functions. The `testAccProviderFactories` global variable is now generically for every test that only requires a single provider and prevents the previous issues where extraneous gRPC plugins were extraneously created (sometimes causing `ulimit` issues, but broadly a performance fix to remove them). This also removes the temporary `testAccProviderFactoriesCur()` and `testAccProviderFactoriesEc2Classic()` functions, since anything only requiring a single provider can use `ProviderFactories: testAccProviderFactories` or `Providers: testAccProviders` (for now, its deprecated in the SDK and we'll be updating these everywhere to `ProviderFactories` anyways). This should mean that other existing EC2-Classic and special service region testing should not require changes to that particular field in future change sets that continue this effort. After this, we are off to the races to remove the problematic environment variable handling causing issues across the provider acceptance testing. Output from acceptance testing (CUR testing done in standalone account): ``` --- PASS: TestAccDataSourceAwsCurReportDefinition_additional (21.63s) --- PASS: TestAccDataSourceAwsCurReportDefinition_basic (18.93s) --- PASS: TestAccDataSourceAwsRoute53ResolverRule_basic (59.47s) --- PASS: TestAccDataSourceAwsRoute53ResolverRule_ResolverEndpointIdWithTags (265.34s) --- PASS: TestAccAWSCloudFrontDistribution_ViewerCertificate_AcmCertificateArn (158.62s) --- PASS: TestAccAWSCloudFrontDistribution_ViewerCertificate_AcmCertificateArn_ConflictsWithCloudFrontDefaultCertificate (159.67s) --- PASS: TestAccAWSCodePipeline_multiregion_basic (37.37s) --- PASS: TestAccAWSCodePipeline_multiregion_Update (58.07s) --- PASS: TestAccAWSCodePipeline_multiregion_ConvertSingleRegion (81.33s) --- PASS: TestAccAwsCurReportDefinition_athena (18.30s) --- PASS: TestAccAwsCurReportDefinition_basic (23.58s) --- PASS: TestAccAwsCurReportDefinition_overwrite (18.10s) --- PASS: TestAccAwsCurReportDefinition_parquet (18.67s) --- PASS: TestAccAwsCurReportDefinition_refresh (21.62s) --- PASS: TestAccAwsCurReportDefinition_textOrCsv (21.45s) --- PASS: TestAccAWSDbSubnetGroupDataSource_basic (66.22s) --- PASS: TestAccAWSDbSubnetGroupDataSource_nonexistent (3.06s) --- PASS: TestAccAwsDxGatewayAssociation_allowedPrefixesVpnGatewayCrossAccount (1998.47s) --- PASS: TestAccAwsDxGatewayAssociation_allowedPrefixesVpnGatewaySingleAccount (1957.89s) --- PASS: TestAccAwsDxGatewayAssociation_basicTransitGatewayCrossAccount (865.40s) --- PASS: TestAccAwsDxGatewayAssociation_basicTransitGatewaySingleAccount (1186.07s) --- PASS: TestAccAwsDxGatewayAssociation_basicVpnGatewayCrossAccount (1530.51s) --- PASS: TestAccAwsDxGatewayAssociation_basicVpnGatewaySingleAccount (1633.65s) --- PASS: TestAccAwsDxGatewayAssociation_multiVpnGatewaysSingleAccount (1697.33s) --- PASS: TestAccAwsDxGatewayAssociation_V0StateUpgrade (1187.97s) --- PASS: TestAccAwsDxGatewayAssociationProposal_AllowedPrefixes (138.31s) --- PASS: TestAccAwsDxGatewayAssociationProposal_basicTransitGateway (227.10s) --- PASS: TestAccAwsDxGatewayAssociationProposal_basicVpnGateway (68.42s) --- PASS: TestAccAwsDxGatewayAssociationProposal_disappears (64.49s) --- SKIP: TestAccAwsDxHostedPrivateVirtualInterface_AccepterTags (0.00s) --- SKIP: TestAccAwsDxHostedPrivateVirtualInterface_basic (0.00s) --- SKIP: TestAccAwsDxHostedPublicVirtualInterface_AccepterTags (0.00s) --- SKIP: TestAccAwsDxHostedPublicVirtualInterface_basic (0.00s) --- PASS: TestAccAwsDxHostedTransitVirtualInterface_serial (0.00s) --- SKIP: TestAccAwsDxHostedTransitVirtualInterface_serial/accepterTags (0.00s) --- SKIP: TestAccAwsDxHostedTransitVirtualInterface_serial/basic (0.00s) --- PASS: TestAccAWSDynamoDbGlobalTable_basic (84.93s) --- PASS: TestAccAWSDynamoDbGlobalTable_multipleRegions (102.52s) --- PASS: TestAccAWSEbsSnapshotCopy_withRegions (75.67s) --- PASS: TestAccAwsRamResourceShareAccepter_basic (36.70s) --- PASS: TestAccAWSRDSCluster_GlobalClusterIdentifier_PrimarySecondaryClusters (1969.60s) --- PASS: TestAccAWSRDSCluster_GlobalClusterIdentifier_ReplicationSourceIdentifier (1819.35s) --- PASS: TestAccAWSRDSCluster_ReplicationSourceIdentifier_KmsKeyId (1691.11s) --- PASS: TestAccAWSRedshiftCluster_snapshotCopy (631.18s) --- PASS: TestAccAWSRedshiftSecurityGroup_basic (11.11s) --- PASS: TestAccAWSRedshiftSecurityGroup_ingressCidr (11.10s) --- PASS: TestAccAWSRedshiftSecurityGroup_ingressSecurityGroup (12.47s) --- PASS: TestAccAWSRedshiftSecurityGroup_updateIngressCidr (23.94s) --- PASS: TestAccAWSRedshiftSecurityGroup_updateIngressSecurityGroup (27.22s) --- PASS: TestAccAWSRoute53VpcAssociationAuthorization_basic (99.00s) --- PASS: TestAccAWSRoute53VpcAssociationAuthorization_disappears (94.43s) --- PASS: TestAccAWSRoute53ZoneAssociation_CrossAccount (124.58s) --- PASS: TestAccAWSRoute53ZoneAssociation_CrossRegion (157.69s) --- PASS: TestAccAWSS3Bucket_Replication (145.26s) --- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation (86.49s) --- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AddAccessControlTranslation (84.75s) --- PASS: TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError (22.79s) --- PASS: TestAccAWSS3Bucket_ReplicationSchemaV2 (174.11s) --- PASS: TestAccAWSS3Bucket_ReplicationWithoutPrefix (65.38s) --- PASS: TestAccAWSS3Bucket_ReplicationWithoutStorageClass (85.05s) --- PASS: TestAccAWSS3Bucket_SameRegionReplicationSchemaV2 (73.58s) --- PASS: TestAccAWSSubnet_ignoreTags (86.24s) --- PASS: TestAccAWSVPCPeeringConnection_accept (144.21s) --- PASS: TestAccAWSVPCPeeringConnection_basic (65.98s) --- PASS: TestAccAWSVPCPeeringConnection_failedState (12.38s) --- PASS: TestAccAWSVPCPeeringConnection_options (117.22s) --- PASS: TestAccAWSVPCPeeringConnection_optionsNoAutoAccept (29.07s) --- PASS: TestAccAWSVPCPeeringConnection_peerRegionAutoAccept (29.13s) --- PASS: TestAccAWSVPCPeeringConnection_plan (52.19s) --- PASS: TestAccAWSVPCPeeringConnection_region (70.00s) --- PASS: TestAccAWSVPCPeeringConnection_tags (150.89s) --- PASS: TestAccAWSVPCPeeringConnectionAccepter_differentRegionDifferentAccount (68.06s) --- PASS: TestAccAWSVPCPeeringConnectionAccepter_differentRegionSameAccount (83.92s) --- PASS: TestAccAWSVPCPeeringConnectionAccepter_sameRegionDifferentAccount (64.85s) --- PASS: TestAccAWSVPCPeeringConnectionAccepter_sameRegionSameAccount (56.44s) --- PASS: TestAccAWSVpcPeeringConnectionOptions_basic (70.42s) --- PASS: TestAccAWSVpcPeeringConnectionOptions_differentRegionSameAccount (75.28s) --- PASS: TestAccAWSVpcPeeringConnectionOptions_sameRegionDifferentAccount (41.96s) --- FAIL: TestAccDataSourceAwsRoute53ResolverRule_SharedByMe (227.22s) --- FAIL: TestAccDataSourceAwsRoute53ResolverRule_SharedWithMe (228.00s) data_source_aws_route53_resolver_rule_test.go:101: Step 1/1 error: Error running apply: 2020/10/28 22:02:11 [DEBUG] Using modified User-Agent: Terraform/0.12.29 HashiCorp-terraform-exec/0.10.0 Error: Error associating principal with RAM resource share: UnknownResourceException: Organization o-upyv668dz5 could not be found. --- FAIL: TestAccAWSEc2TransitGatewayVpcAttachment_SharedTransitGateway (221.84s) resource_aws_ec2_transit_gateway_vpc_attachment_test.go:215: Step 1/2 error: Error running apply: 2020/10/28 22:02:10 [DEBUG] Using modified User-Agent: Terraform/0.12.29 HashiCorp-terraform-exec/0.10.0 Error: Error associating principal with RAM resource share: UnknownResourceException: Organization o-upyv668dz5 could not be found. Error: error associating RAM Resource Share: OperationNotPermittedException: The resource you are attempting to share can only be shared within your AWS Organization. This error may also occur if you have not enabled sharing with your AWS organization, or that onboarding process is still in progress. --- FAIL: TestAccAWSEc2TransitGatewayVpcAttachmentAccepter_Tags (201.45s) --- FAIL: TestAccAWSEc2TransitGatewayVpcAttachmentAccepter_TransitGatewayDefaultRouteTableAssociationAndPropagation (211.45s) --- FAIL: TestAccAWSEc2TransitGatewayVpcAttachmentAccepter_basic (221.42s) resource_aws_ec2_transit_gateway_vpc_attachment_accepter_test.go:135: Step 1/4 error: Error running apply: 2020/10/28 22:02:11 [DEBUG] Using modified User-Agent: Terraform/0.12.29 HashiCorp-terraform-exec/0.10.0 Error: error associating RAM Resource Share: OperationNotPermittedException: The resource you are attempting to share can only be shared within your AWS Organization. This error may also occur if you have not enabled sharing with your AWS organization, or that onboarding process is still in progress. Error: Error associating principal with RAM resource share: OperationNotPermittedException: The resource you are attempting to share can only be shared within your AWS Organization. This error may also occur if you have not enabled sharing with your AWS organization, or that onboarding process is still in progress. ```
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some notes for the reviewer - please let me know if you have questions.
@@ -7,18 +7,16 @@ import ( | |||
|
|||
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/acctest" | |||
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource" | |||
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These tests did not require extra providers or saving them.
@@ -12,7 +12,6 @@ import ( | |||
"github.com/aws/aws-sdk-go/service/cloudfront" | |||
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/acctest" | |||
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource" | |||
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Similarly here, this testing did not need extra providers or to save the instances.
@@ -272,16 +269,14 @@ resource "aws_dynamodb_global_table" "test" { | |||
} | |||
|
|||
func testAccDynamoDbGlobalTableConfig_invalidName(tableName string) string { | |||
return composeConfig(testAccAlternateRegionProviderConfig(), fmt.Sprintf(` | |||
data "aws_region" "alternate" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wasn't necessary. 😄
testAccMultipleRegionPreCheck(t, 2) | ||
testAccPreCheckAWSRdsGlobalCluster(t) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Previously missing PreCheck for this configuration.
provider := providerF() | ||
|
||
conn := provider.Meta().(*AWSClient).s3conn | ||
conn := testAccProvider.Meta().(*AWSClient).s3conn |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We previously did and always should reference the main provider instance in this type of testing for simplicity.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great!
This has been released in version 3.14.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks! |
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks! |
Community Note
Reference: #8316
Reference: #15737
Reference: #15791
Release note for CHANGELOG:
This change set continues the work of #15877, bringing a consistent pattern for cross-region and cross-account testing and removing previously deprecated provider-level test functions. The
testAccProviderFactories
global variable is now generically for every test that only requires a single provider and prevents the previous issues where extraneous gRPC plugins were extraneously created (sometimes causingulimit
issues, but broadly a performance fix to remove them).This also removes the temporary
testAccProviderFactoriesCur()
andtestAccProviderFactoriesEc2Classic()
functions, since anything only requiring a single provider can useProviderFactories: testAccProviderFactories
orProviders: testAccProviders
(for now, its deprecated in the SDK and we'll be updating these everywhere toProviderFactories
anyways). This should mean that other existing EC2-Classic and special service region testing should not require changes to that particular field in future change sets that continue this effort.After this, we are off to the races to remove the problematic environment variable handling causing issues across the provider acceptance testing.
Output from acceptance testing (CUR testing done in standalone account):