[Bug]: Failed to marshal plan to json: eror marshaling prior state

[jgrumboe]

Terraform Core Version

1.7.5

AWS Provider Version

5.59.0

Affected Resource(s)

• aws_elasticache_replication_group

Expected Behavior

I upgrade the provider from 4.67.0 to 5.59.0 and see a successful plan.

Actual Behavior

I upgrade the provider from 4.67.0 to 5.59.0 and the plan fails with the following error:

╷
│ Error: string is required
│ 
│   with module.redis_blue_shared.aws_elasticache_replication_group.default,
│   on .terraform/modules/redis_blue_shared/main.tf line 43, in resource "aws_elasticache_replication_group" "default":
│   43: resource "aws_elasticache_replication_group" "default" {
│ 
╵
Failed generating plan JSON
Exit code: 1

Failed to marshal plan to json: error marshaling prior state: schema version 1 for aws_elasticache_replication_group.default in state does not match version 2 from the provider
Operation failed: 2 errors occurred:
        * failed running terraform plan (exit 1)
        * failed generating plan JSON: failed running command (exit 1)

As a second test I upgrade from 4.67.0 to 5.58.0 and the plan is successful!

Relevant Error/Panic Output Snippet

The module reference in my code is just creating a normal `aws_elasticache_replication_group`. I'd say no magic here.


resource "aws_elasticache_replication_group" "default" {
  replication_group_id        = local.name
  description                 = local.group_description
  node_type                   = var.instance_type
  num_cache_clusters          = var.num_cache_clusters
  engine_version              = var.engine_version
  transit_encryption_enabled  = var.transit_encryption_enabled
  auth_token                  = (var.transit_encryption_enabled && var.auth_enabled) ? random_password.auth_token[0].result : null
  parameter_group_name        = join("", aws_elasticache_parameter_group.default.*.name)
  maintenance_window          = "wed:03:00-wed:04:00"
  automatic_failover_enabled  = var.automatic_failover_enabled
  port                        = 6379
  snapshot_retention_limit    = 0
  at_rest_encryption_enabled  = false
  apply_immediately           = true
  multi_az_enabled            = var.multi_az_enabled
  preferred_cache_cluster_azs = slice(var.availability_zones, 0, var.num_cache_clusters)
  subnet_group_name           = aws_elasticache_subnet_group.default.name
  security_group_ids          = [aws_security_group.default.id]

  tags = merge(
    local.common_tags,
    {
      Name = local.group_description,
    }
  )
  lifecycle {
    ignore_changes = [
      replication_group_id
    ]
  }
}

### Terraform Configuration Files

I haven't created one yet. Will do so and add it.

### Steps to Reproduce

* have a working `aws_elasticache_replication_group` created with AWS provider version 4.67.0
* Upgrade AWS provider to 5.59.0
* run `terraform plan` and it will fail with the marshaling error

* upgrade AWS provider 5.58.0
* run `terraform plan` and it will succeed

### Debug Output

_No response_

### Panic Output

_No response_

### Important Factoids

_No response_

### References

_No response_

### Would you like to implement a fix?

None

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[ewbankkit]

I can reproduce with a new acceptance test case:

% ACCTEST_TIMEOUT=1080m make testacc TESTARGS='-run=TestAccElastiCacheReplicationGroup_stateUpgrade5590' PKG=elasticache
make: Verifying source code with gofmt...
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.22.5 test ./internal/service/elasticache/... -v -count 1 -parallel 20  -run=TestAccElastiCacheReplicationGroup_stateUpgrade5590 -timeout 1080m
=== RUN   TestAccElastiCacheReplicationGroup_stateUpgrade5590
=== PAUSE TestAccElastiCacheReplicationGroup_stateUpgrade5590
=== CONT  TestAccElastiCacheReplicationGroup_stateUpgrade5590
    replication_group_test.go:657: Step 2/2 error: Error running pre-apply plan: exit status 1
        
        Error: string is required
        
          with aws_elasticache_replication_group.test,
          on terraform_plugin_test.tf line 12, in resource "aws_elasticache_replication_group" "test":
          12: resource "aws_elasticache_replication_group" "test" {
        
    panic.go:626: Error retrieving state, there may be dangling resources: exit status 1
        Failed to marshal state to json: schema version 1 for aws_elasticache_replication_group.test in state does not match version 2 from the provider
--- FAIL: TestAccElastiCacheReplicationGroup_stateUpgrade5590 (540.76s)
FAIL
FAIL	github.com/hashicorp/terraform-provider-aws/internal/service/elasticache	545.704s
FAIL
make: *** [testacc] Error 1

Relates #38046.

Detailed log:

2024-07-22T16:07:39.133-0400 [DEBUG] sdk.helper_schema: attribute no longer present in schema: tf_resource_type=aws_elasticache_replication_group tf_mux_provider="*schema.GRPCProviderServer" attribute=availability_zones tf_rpc=UpgradeResourceState tf_req_id=f6ca157a-e589-7a7d-e123-14ac4d002e9e tf_provider_addr=registry.terraform.io/hashicorp/aws
2024-07-22T16:07:39.133-0400 [DEBUG] sdk.helper_schema: attribute no longer present in schema: tf_resource_type=aws_elasticache_replication_group tf_mux_provider="*schema.GRPCProviderServer" tf_rpc=UpgradeResourceState tf_req_id=f6ca157a-e589-7a7d-e123-14ac4d002e9e tf_provider_addr=registry.terraform.io/hashicorp/aws attribute=number_cache_clusters
2024-07-22T16:07:39.133-0400 [DEBUG] sdk.helper_schema: attribute no longer present in schema: tf_mux_provider="*schema.GRPCProviderServer" attribute=replication_group_description tf_rpc=UpgradeResourceState tf_req_id=f6ca157a-e589-7a7d-e123-14ac4d002e9e tf_provider_addr=registry.terraform.io/hashicorp/aws tf_resource_type=aws_elasticache_replication_group
2024-07-22T16:07:39.133-0400 [ERROR] sdk.proto: Response contains error diagnostic: tf_resource_type=aws_elasticache_replication_group tf_req_id=f6ca157a-e589-7a7d-e123-14ac4d002e9e tf_provider_addr=registry.terraform.io/hashicorp/aws tf_rpc=UpgradeResourceState diagnostic_severity=ERROR diagnostic_summary="string is required" diagnostic_attribute="AttributeName(\"cluster_mode\")" diagnostic_detail="" tf_proto_version=5.6

Relates #38002.

Actually, the most likely suspect is the change to make description Required.
Relates #38360.
Relates #31008.

The v4.67.0 schema contained block attribute named cluster_mode. It was removed at v5.0.0 (#31008).
The v5.59.0 schema introduced a new string attribute named cluster_mode.
It's this reintroduction that causes the state upgrade error.

[github-actions]

Warning

This issue has been closed, meaning that any additional comments are hard for our team to see. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

[github-actions]

This functionality has been released in v5.60.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.