[Bug]: When trying to create a tag with an empty string as value, other tags are deleted. (provider version >= 5.0.0) #31941

suminhong · 2023-06-14T05:55:37Z

Terraform Core Version

1.4.5, 1.5.0

AWS Provider Version

>= 5.0.0

Affected Resource(s)

ALL

Expected Behavior

When adding a tag with an empty string, I think it shouldn't affect an existing tag, even if the tag isn't actually being created.

Actual Behavior

From aws provider version 5.0.0 or later, when adding a tag with an empty string value, no tag information is reflected in tags_all, and all existing tags disappear when actually applied.

When apply is actually run for the above Plans, new tags are not visible and existing tags are deleted, as shown in tags_all.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

provider "aws" {
  region  = local.aws_region
  profile = local.aws_profile

  assume_role {
    role_arn = local.assume_role["prod"]
  }

  alias = "prod"

  ignore_tags {
    key_prefixes = ["aws-cdk:"]
  }
}

resource "aws_s3_bucket" "test" {
  bucket = "wad-test-analytics-data"
  tags   = merge(local.env_tag, { Name = "wad-test-analytics-data", test = "" })

  provider = aws.prod
}

Steps to Reproduce

Upgrade the aws provider version to 5.0.0 or higher.
Add {any=""} to the tag of any resource.
Plan and apply execution

Debug Output

No response

Panic Output

No response

Important Factoids

I tested it in aws_s3_bucket, aws_security_group.

References

No response

Would you like to implement a fix?

None

The text was updated successfully, but these errors were encountered:

github-actions · 2023-06-14T05:55:50Z

Community Note

Voting for Prioritization

Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
Please see our prioritization guide for information on how we prioritize.
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

If you are interested in working on this issue, please leave a comment.
If this would be your first contribution, please review the contribution guide.

tom-dalton-fanduel · 2023-06-20T15:20:24Z

This appears to be more widespread than just S3 - I've also seen this issue affecting the following resources:

aws_iam_role
aws_cloudfront_distribution
aws_cloudwatch_log_group
aws_lambda_function

Presence of tags are used for all sorts of things, so having TF silently deleting them all is quite impactful. The TF plan says it's going to add tags, the apply succeeds and says:

aws_s3_bucket.default: Modifying... [id=****]
...
aws_s3_bucket.default: Modifications complete after 1s [id=****]

but then all the tags on the resource (existing and new) are deleted.

wad-hongsumin · 2023-06-24T03:54:40Z

Thanks for the additional testimony. However, this problem is happening in all resources, and there seems to be a bug in the process of setting tags_all during the version transition from 4.67 to 5.0.
I can't go through all the code, but I'm guessing it's probably a bug from the code changes below.
v4.67.0...v5.0.0#diff-f321370774c4f8e1fd4ac201a11bc6c17a4e4be89c74a19f875a12f17754ee26L92-L103

oarmstrong · 2023-06-30T12:54:21Z

This doesn't seem to be an issue with all resources. I can replicate the issue with aws_s3_bucket but not with aws_sns_topic. SNS topics do appear to correctly allow tags with an empty string as the value. The original issue that documented this is #24449 which is sadly locked so I cannot reference this there, but that does not appear to be working correctly. I did run these tests on version 5.0.0 as well to see if it was a regression since that release but I got the same results.

A particularly interesting INFO log line is (Project being the tag I've been testing with):

2023-06-30T13:34:24.385+0100 [WARN]  Provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an unexpected new value for aws_s3_bucket.this, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .tags: element "Project" has vanished

Full reproduction code and output below. You can see that everything applies correctly, and on the second plan it attempts again to add the tag to the bucket but not to the SNS topic. I have redacted my AWS account ID to 999999999.

# main.tf
resource "aws_s3_bucket" "this" {
  bucket = "4c59cf5d-0310-4b37-be46-a34e90f065f1"
  tags = {
    Project = ""
  }
}

resource "aws_sns_topic" "this" {
  name = "4fa33a6b-2c7a-41c9-9f05-e30176276ce8"
  tags = {
    Project = ""
  }
}

$ terraform version
Terraform v1.5.2
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v5.6.1

$ TF_LOG=info terraform apply -auto-approve
2023-06-30T13:34:18.803+0100 [INFO]  Terraform version: 1.5.2
2023-06-30T13:34:18.803+0100 [INFO]  Go runtime version: go1.20
2023-06-30T13:34:18.803+0100 [INFO]  CLI args: []string{"/home/oca/bin/tfenv/versions/1.5.2/terraform", "apply", "-auto-approve"}
2023-06-30T13:34:18.803+0100 [INFO]  CLI command args: []string{"apply", "-auto-approve"}
2023-06-30T13:34:19.725+0100 [INFO]  backend/local: starting Apply operation
2023-06-30T13:34:19.726+0100 [INFO]  provider: configuring client automatic mTLS
2023-06-30T13:34:19.865+0100 [INFO]  provider.terraform-provider-aws_v5.6.1_x5: configuring server automatic mTLS: timestamp=2023-06-30T13:34:19.865+0100
2023-06-30T13:34:20.573+0100 [INFO]  provider: configuring client automatic mTLS
2023-06-30T13:34:20.738+0100 [INFO]  provider.terraform-provider-aws_v5.6.1_x5: configuring server automatic mTLS: timestamp=2023-06-30T13:34:20.737+0100
2023-06-30T13:34:21.275+0100 [INFO]  backend/local: apply calling Plan
2023-06-30T13:34:21.276+0100 [INFO]  provider: configuring client automatic mTLS
2023-06-30T13:34:21.433+0100 [INFO]  provider.terraform-provider-aws_v5.6.1_x5: configuring server automatic mTLS: timestamp=2023-06-30T13:34:21.432+0100
2023-06-30T13:34:22.005+0100 [INFO]  provider.terraform-provider-aws_v5.6.1_x5: Retrieved credentials: @module=aws.aws-base tf_aws.credentials_source=EnvConfigCredentials tf_provider_addr=registry.terraform.io/hashicorp/aws @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/logger.go:33 tf_mux_provider=*schema.GRPCProviderServer tf_req_id=0a284871-c95f-2b58-a418-d088e5f6cffd tf_rpc=ConfigureProvider timestamp=2023-06-30T13:34:22.005+0100
2023-06-30T13:34:22.148+0100 [INFO]  provider.terraform-provider-aws_v5.6.1_x5: Retrieved caller identity from STS: tf_provider_addr=registry.terraform.io/hashicorp/aws @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/logger.go:33 @module=aws.aws-base tf_mux_provider=*schema.GRPCProviderServer tf_req_id=0a284871-c95f-2b58-a418-d088e5f6cffd tf_rpc=ConfigureProvider timestamp=2023-06-30T13:34:22.147+0100
2023-06-30T13:34:22.173+0100 [INFO]  provider.terraform-provider-aws_v5.6.1_x5: Retrieved caller identity from STS: tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=0a284871-c95f-2b58-a418-d088e5f6cffd @module=aws.aws-base tf_mux_provider=*schema.GRPCProviderServer tf_rpc=ConfigureProvider @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/logger.go:33 timestamp=2023-06-30T13:34:22.173+0100
2023-06-30T13:34:22.189+0100 [WARN]  Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for aws_sns_topic.this, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .fifo_topic: planned value cty.False for a non-computed attribute
      - .content_based_deduplication: planned value cty.False for a non-computed attribute
2023-06-30T13:34:22.197+0100 [WARN]  Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for aws_s3_bucket.this, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .force_destroy: planned value cty.False for a non-computed attribute
      - .versioning: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .cors_rule: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .replication_configuration: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .logging: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .object_lock_configuration: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .server_side_encryption_configuration: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .website: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .grant: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .lifecycle_rule: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_s3_bucket.this will be created
  + resource "aws_s3_bucket" "this" {
      + acceleration_status         = (known after apply)
      + acl                         = (known after apply)
      + arn                         = (known after apply)
      + bucket                      = "4c59cf5d-0310-4b37-be46-a34e90f065f1"
      + bucket_domain_name          = (known after apply)
      + bucket_prefix               = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = false
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + object_lock_enabled         = (known after apply)
      + policy                      = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags                        = {
          + "Project" = ""
        }
      + tags_all                    = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)
    }

  # aws_sns_topic.this will be created
  + resource "aws_sns_topic" "this" {
      + arn                         = (known after apply)
      + content_based_deduplication = false
      + fifo_topic                  = false
      + id                          = (known after apply)
      + name                        = "4fa33a6b-2c7a-41c9-9f05-e30176276ce8"
      + name_prefix                 = (known after apply)
      + owner                       = (known after apply)
      + policy                      = (known after apply)
      + signature_version           = (known after apply)
      + tags                        = {
          + "Project" = ""
        }
      + tags_all                    = (known after apply)
      + tracing_config              = (known after apply)
    }

Plan: 2 to add, 0 to change, 0 to destroy.
2023-06-30T13:34:22.288+0100 [INFO]  backend/local: apply calling Apply
2023-06-30T13:34:22.289+0100 [INFO]  provider: configuring client automatic mTLS
2023-06-30T13:34:22.439+0100 [INFO]  provider.terraform-provider-aws_v5.6.1_x5: configuring server automatic mTLS: timestamp=2023-06-30T13:34:22.439+0100
2023-06-30T13:34:23.004+0100 [INFO]  provider.terraform-provider-aws_v5.6.1_x5: Retrieved credentials: @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/logger.go:33 tf_aws.credentials_source=EnvConfigCredentials tf_req_id=9c8996d2-b998-e4e9-6b8b-55349852476b @module=aws.aws-base tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/hashicorp/aws tf_rpc=ConfigureProvider timestamp=2023-06-30T13:34:23.004+0100
2023-06-30T13:34:23.140+0100 [INFO]  provider.terraform-provider-aws_v5.6.1_x5: Retrieved caller identity from STS: tf_rpc=ConfigureProvider @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/logger.go:33 @module=aws.aws-base tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=9c8996d2-b998-e4e9-6b8b-55349852476b timestamp=2023-06-30T13:34:23.140+0100
2023-06-30T13:34:23.164+0100 [INFO]  provider.terraform-provider-aws_v5.6.1_x5: Retrieved caller identity from STS: @caller=github.com/hashicorp/aws-sdk-go-base/[email protected]/logging/logger.go:33 tf_mux_provider=*schema.GRPCProviderServer tf_req_id=9c8996d2-b998-e4e9-6b8b-55349852476b @module=aws.aws-base tf_provider_addr=registry.terraform.io/hashicorp/aws tf_rpc=ConfigureProvider timestamp=2023-06-30T13:34:23.164+0100
2023-06-30T13:34:23.186+0100 [WARN]  Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for aws_sns_topic.this, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .fifo_topic: planned value cty.False for a non-computed attribute
      - .content_based_deduplication: planned value cty.False for a non-computed attribute
aws_sns_topic.this: Creating...
2023-06-30T13:34:23.186+0100 [INFO]  Starting apply for aws_sns_topic.this
2023-06-30T13:34:23.197+0100 [WARN]  Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for aws_s3_bucket.this, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .force_destroy: planned value cty.False for a non-computed attribute
      - .cors_rule: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .replication_configuration: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .versioning: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .website: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .grant: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .lifecycle_rule: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .logging: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .object_lock_configuration: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
      - .server_side_encryption_configuration: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
aws_s3_bucket.this: Creating...
2023-06-30T13:34:23.197+0100 [INFO]  Starting apply for aws_s3_bucket.this
2023-06-30T13:34:23.651+0100 [WARN]  Provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an unexpected new value for aws_sns_topic.this, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .http_success_feedback_sample_rate: was null, but now cty.NumberIntVal(0)
      - .sqs_success_feedback_sample_rate: was null, but now cty.NumberIntVal(0)
      - .lambda_failure_feedback_role_arn: was null, but now cty.StringVal("")
      - .http_failure_feedback_role_arn: was null, but now cty.StringVal("")
      - .sqs_success_feedback_role_arn: was null, but now cty.StringVal("")
      - .firehose_failure_feedback_role_arn: was null, but now cty.StringVal("")
      - .firehose_success_feedback_sample_rate: was null, but now cty.NumberIntVal(0)
      - .lambda_success_feedback_sample_rate: was null, but now cty.NumberIntVal(0)
      - .sqs_failure_feedback_role_arn: was null, but now cty.StringVal("")
      - .application_success_feedback_role_arn: was null, but now cty.StringVal("")
      - .application_success_feedback_sample_rate: was null, but now cty.NumberIntVal(0)
      - .delivery_policy: was null, but now cty.StringVal("")
      - .http_success_feedback_role_arn: was null, but now cty.StringVal("")
      - .firehose_success_feedback_role_arn: was null, but now cty.StringVal("")
      - .application_failure_feedback_role_arn: was null, but now cty.StringVal("")
      - .display_name: was null, but now cty.StringVal("")
      - .kms_master_key_id: was null, but now cty.StringVal("")
      - .lambda_success_feedback_role_arn: was null, but now cty.StringVal("")
aws_sns_topic.this: Creation complete after 1s [id=arn:aws:sns:eu-west-1:999999999999:4fa33a6b-2c7a-41c9-9f05-e30176276ce8]
2023-06-30T13:34:24.322+0100 [WARN]  provider.terraform-provider-aws_v5.6.1_x5: [WARN] Unable to read S3 bucket (4c59cf5d-0310-4b37-be46-a34e90f065f1) Object Lock Configuration: ObjectLockConfigurationNotFoundError: Object Lock configuration does not exist for this bucket
2023-06-30T13:34:24.385+0100 [WARN]  Provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an unexpected new value for aws_s3_bucket.this, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .tags: element "Project" has vanished
aws_s3_bucket.this: Creation complete after 1s [id=4c59cf5d-0310-4b37-be46-a34e90f065f1]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

$ terraform plan
aws_sns_topic.this: Refreshing state... [id=arn:aws:sns:eu-west-1:999999999999:4fa33a6b-2c7a-41c9-9f05-e30176276ce8]
aws_s3_bucket.this: Refreshing state... [id=4c59cf5d-0310-4b37-be46-a34e90f065f1]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_s3_bucket.this will be updated in-place
  ~ resource "aws_s3_bucket" "this" {
        id                          = "4c59cf5d-0310-4b37-be46-a34e90f065f1"
      ~ tags                        = {
          + "Project" = ""
        }
      ~ tags_all                    = {} -> (known after apply)
        # (9 unchanged attributes hidden)

        # (3 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

$ aws s3api get-bucket-tagging --bucket 4c59cf5d-0310-4b37-be46-a34e90f065f1

An error occurred (NoSuchTagSet) when calling the GetBucketTagging operation: The TagSet does not exist

$ aws sns list-tags-for-resource --resource-arn arn:aws:sns:eu-west-1:999999999999:4fa33a6b-2c7a-41c9-9f05-e30176276ce8
{
    "Tags": [
        {
            "Key": "Project",
            "Value": ""
        }
    ]
}

oarmstrong · 2023-06-30T14:03:42Z

A bit of further information, it seems like the bug was originally introduced in v3.38.0. In v3.37.0 it performs as expected, adding the empty tag to both S3 buckets and SNS topics (I'm assuming all resources but I've only tested those two). In v3.38.0 I get the same behaviour as explained above. I did not test every version in between v3.38.0 and v5.6.1 but those that I did test also showed the same broken behaviour.

So in my understanding v3.38.0 introduced the bug and v5.0.0 did not fix it as #24449 suggests it did.

At first glance of the changelog for v3.38.0 the most likely candidates could be #18958 or #19084. I may attempt to bisect through the commits to confirm if I find the time.

oarmstrong · 2023-06-30T14:44:39Z

I have managed to run through a bisect and identified the commit that introduced the issue to be 0578e55. I'm aware that a lot has changed in the design of the repository since then and #24449 did something that attempted to fix the issue.

I think I'm at my limit here, that commit is massive and I don't think I can identify the culprit. I keep thinking about it but good luck to whoever can manage to dive into that.

I hope this isn't considered rude, tagging those that seem to be involved in that issue so perhaps they may be able to chime in. Much appreciated if you can help, folks @petarlishov @justinretzolk @johnsonaj .

johnsonaj · 2023-06-30T17:22:26Z

Hi @oarmstrong, thank you for the investigation and updates! With the release of v5.0.0 the maintainers made quite a few changes to how tagging is done in the provider. These changes allowed a more transparent tagging mechanism for all resources, and also implemented some fixes for default_tags and tags, including having empty values.

The AWS API implements resource tagging pretty generically across most services, however there are a few resources where the provider implements custom tagging functions, allowing it to work correctly. These services include resources in the following:

S3
IAM
Cloudwatch (Logs)
Service Catalog
EC2
Inspector

We are working on a plan to update and include these resources in the new tagging process. At this moment we do not have a projected ETA but we will keep this issue updated with any progress/PRs.

oarmstrong · 2023-07-01T18:18:07Z

Thanks a lot for the info. That makes sense, it was fixed for all resources except those you listed. Worth noting that a few comments up it was pointed out that Lambda Functions seem to exhibit this issue as well and aren't included in your list (#31941 (comment)). I haven't independently verified that myself.

Is there a place where these discussions are occurring for these resources? I wouldn't mind taking a stab at resolving this for S3 buckets (as that's what's currently blocking me at work) but I don't want to waste effort if it doesn't line up with the plan for the bigger picture or if that work is already in progress by someone else.

ghost · 2023-07-12T02:29:51Z

Hi! 👋

I want to add ABAC exists: https://docs.aws.amazon.com/organizations/latest/userguide/orgs_tagging_abac.html

This bug could actually be the source of big dysfunctions or security issues if people applies inadvertently.

Big blocker in my case :(

Thanks!

Prashanteppalpalli · 2023-08-21T11:32:50Z

Hi There,

Is there any update on the fix ?
I am trying to add tags on s3 bucket. Some of values are empty due to which its not updating any tag. But if I remove those empty values then only its updating tags (where actual key and values are present)

tonyfosterdev · 2023-10-07T18:12:10Z

I'm seeing this with aws_iam_role resources when using Hashicorp's ECS Mesh-Init task for ECS. When not using a consul namespace, an empty string is used as a tag value. We didn't notice this prior to v5, but I don't know when the exact start was.

From our plan, you can see the tags need to be added:

  # module.core.module.consul_service_definition.aws_iam_role.task[0] will be updated in-place
  ~ resource "aws_iam_role" "task" {
        id                    = "my-service-task"
        name                  = "my-service-task"
      ~ tags                  = {
          + "consul.hashicorp.com.namespace"    = ""
          + "consul.hashicorp.com.service-name" = "my-service"
        }
      ~ tags_all              = {} -> (known after apply)
        # (8 unchanged attributes hidden)
    }

In our logs:

2023-10-07T17:49:56.827Z [WARN]  Provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an unexpected new value for module.core.module.consul_service_definition.aws_iam_role.task[0], but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .tags: element "consul.hashicorp.com.namespace" has vanished
      - .tags: element "consul.hashicorp.com.service-name" has vanished
module.core.module.consul_service_definition.aws_iam_role.task[0]: Modifications complete after 0s [id="my-service-task]

I'm currently blocked by this as I can't get my service to connect to consul now. I'm also not sure how to revert to an old provider version at this point. If anyone is aware of a work around, I'd appreciate it.

boreasoss · 2023-11-21T14:43:32Z

Additional resources that are affected:

AWS CloudTrail trails
AWS Route53 resolver endpoints

A workaround I found that got me past the line is to simply use the null type instead of empty strings and this gets past the v5.x.x provider version bug. You can't downgrade the provider version so this seemed to be a better workaround than supplying a dummy value which may cause issues of its own.

As the documentation suggests, a null type acts as an omission so the offending tag in question won't get passed to the tag map tags_all at all:

null: a value that represents absence or omission. If you set an argument of a resource to null, Terraform behaves as though you had completely omitted it — it will use the argument's default value if it has one, or raise an error if the argument is mandatory. null is most useful in conditional expressions, so you can dynamically omit an argument if a condition isn't met.

ref: Types and Values

mikeizzy · 2023-12-17T11:48:11Z

Was facing this issue when applying empty string tags at any level be it:

default_tags
local.common_tags
tags on the resource itself

I've found the issue goes away and get an No changes. Your infrastructure matches the configuration. when you remove all empty string tag values, i.e. ones like some_random_empty_tag = ""

danielw-aws · 2024-01-04T21:39:54Z

I spent some time over the holiday break digging into this. I set up a bunch of tests in multiple places that I thought this could be and was able to rule out any code inside of the AWS Terraform Provider. I ultimately traced this bug to a function call in /internal/verify/diff.go in the SetTagsDiff function that starts at line 24.

I wired up the code to identify where and how the data was being manipulated and ultimately found that this may be a bug in the upstream terraform-plugin-sdk. I have a forked branch that I set up a print statements before and after line 50 which looks like the following:

fmt.Println("[Triage] diff.go func SetTagsDiff before SetNewComputed ,diff.Get(names.AttrTags)=", diff.Get(names.AttrTags))
fmt.Println("[Triage] diff.go func SetTagsDiff before SetNewComputed, diff.Get(names.AttrTagsAll)=", diff.Get(names.AttrTagsAll))
if newTags.HasZeroValue() {
	fmt.Println("[Triage] diff.go newTags.HasZeroValue()=", newTags.HasZeroValue())
	if err := diff.SetNewComputed("tags_all"); err != nil {
		return fmt.Errorf("error setting tags_all to computed: %w", err)
	}
}
fmt.Println("[Triage] diff.go func SetTagsDiff after SetNewComputed, diff.Get(names.AttrTagsAll)=", diff.Get(names.AttrTagsAll))
fmt.Println("[Triage] diff.go func SetTagsDiff after SetNewComputed ,diff.Get(names.AttrTags)=", diff.Get(names.AttrTags))

and wrote a test that captured the above use case of a populated tag to null tag which then translated to this output when the test ran:

[Triage] diff.go func SetTagsDiff before SetNewComputed ,diff.Get(names.AttrTags)= map[Key2: Key3:XXX Key4:DDD Key5:EEE]
[Triage] diff.go func SetTagsDiff before SetNewComputed, diff.Get(names.AttrTagsAll)= map[Key2:BBB Key3:XXX Key4:DDD Key5:EEE]
[Triage] diff.go newTags.HasZeroValue()= true
[Triage] diff.go func SetTagsDiff after SetNewComputed, diff.Get(names.AttrTagsAll)= map[]
[Triage] diff.go func SetTagsDiff after SetNewComputed ,diff.Get(names.AttrTags)= map[Key2: Key3:XXX Key4:DDD Key5:EEE]

The expected output of a null tag failed due to the return of an empty map.

I am going to attempt to find out what is happening in SetNewComputed in the terraform-plugin-sdk that is causing the function to return an empty map which seems to be the root of this issue.

elyabesantos · 2024-02-02T20:22:55Z

+1

…t. Known terraform issue. See hashicorp/terraform-provider-aws#31941

…t. Known terraform issue. See hashicorp/terraform-provider-aws#31941 (#62)

github-actions · 2024-03-07T23:07:39Z

This functionality has been released in v5.40.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

github-actions · 2024-04-07T02:03:51Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

suminhong added bug Addresses a defect in current functionality. needs-triage Waiting for first response or review from a maintainer. labels Jun 14, 2023

github-actions bot added the service/s3 Issues and PRs that pertain to the s3 service. label Jun 14, 2023

ewbankkit added the tags Pertains to resource tagging. label Jun 14, 2023

justinretzolk removed the needs-triage Waiting for first response or review from a maintainer. label Jun 14, 2023

johnsonaj mentioned this issue Jul 3, 2023

lambda: add ListTags to allow computed tags #32348

Merged

maunzCache mentioned this issue Jul 20, 2023

[Bug]: aws_s3_bucket resource incosistent with provider.aws.default_tags #32618

Closed

johnsonaj mentioned this issue Aug 3, 2023

Error: Provider produced inconsistent final plan - tags_all error when using default_tags block #24008

Closed

ewbankkit mentioned this issue Aug 29, 2023

tags: send all old and new tags to updateTags #33226

Merged

t0yv0 mentioned this issue Oct 18, 2023

Incorrect tags applied when tag is empty pulumi/pulumi-aws#2895

Closed

johnsonaj mentioned this issue Oct 30, 2023

[Bug]: Empty Tag in Provider's default_tags Clears Previous Tags #34155

Open

ewbankkit mentioned this issue Oct 31, 2023

[Bug]: Adding Tags to S3 Buckets Causes Perpetual Drift #34185

Closed

gdavison self-assigned this Feb 7, 2024

terraform-aws-provider bot added the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Feb 7, 2024

gdavison mentioned this issue Feb 8, 2024

Allow empty values for tags for S3 resources #35710

Merged

Kjaamor added a commit to ministryofjustice/hmpps-ems-platform-terraform-modules that referenced this issue Feb 9, 2024

Bugfix where default values cannot be empty. Workaround to add conten…

1027cab

…t. Known terraform issue. See hashicorp/terraform-provider-aws#31941

Kjaamor mentioned this issue Feb 9, 2024

ELM-1513: Bugfix where default values cannot be empty. Workaround to add content ministryofjustice/hmpps-ems-platform-terraform-modules#62

Merged

Kjaamor added a commit to ministryofjustice/hmpps-ems-platform-terraform-modules that referenced this issue Feb 9, 2024

Bugfix where default values cannot be empty. Workaround to add conten…

e460d5f

…t. Known terraform issue. See hashicorp/terraform-provider-aws#31941 (#62)

ewbankkit mentioned this issue Feb 9, 2024

IAM: Participate in transparent tagging #35740

Merged

fcastagninizoom mentioned this issue Feb 16, 2024

[Bug]: Inconsistent Application of Dynamic Default Tags to Resources #35725

Closed

gdavison closed this as completed in #35710 Mar 4, 2024

github-actions bot added this to the v5.40.0 milestone Mar 4, 2024

github-actions bot removed the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Mar 7, 2024

github-actions bot locked as resolved and limited conversation to collaborators Apr 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: When trying to create a tag with an empty string as value, other tags are deleted. (provider version >= 5.0.0) #31941

[Bug]: When trying to create a tag with an empty string as value, other tags are deleted. (provider version >= 5.0.0) #31941

suminhong commented Jun 14, 2023 •

edited

Loading

github-actions bot commented Jun 14, 2023

tom-dalton-fanduel commented Jun 20, 2023 •

edited

Loading

wad-hongsumin commented Jun 24, 2023

oarmstrong commented Jun 30, 2023

oarmstrong commented Jun 30, 2023 •

edited

Loading

oarmstrong commented Jun 30, 2023 •

edited

Loading

johnsonaj commented Jun 30, 2023

oarmstrong commented Jul 1, 2023

ghost commented Jul 12, 2023

Prashanteppalpalli commented Aug 21, 2023 •

edited

Loading

tonyfosterdev commented Oct 7, 2023 •

edited

Loading

boreasoss commented Nov 21, 2023

mikeizzy commented Dec 17, 2023

danielw-aws commented Jan 4, 2024

elyabesantos commented Feb 2, 2024

github-actions bot commented Mar 7, 2024

github-actions bot commented Apr 7, 2024

[Bug]: When trying to create a tag with an empty string as value, other tags are deleted. (provider version >= 5.0.0) #31941

[Bug]: When trying to create a tag with an empty string as value, other tags are deleted. (provider version >= 5.0.0) #31941

Comments

suminhong commented Jun 14, 2023 • edited Loading

Terraform Core Version

AWS Provider Version

Affected Resource(s)

Expected Behavior

Actual Behavior

Relevant Error/Panic Output Snippet

Terraform Configuration Files

Steps to Reproduce

Debug Output

Panic Output

Important Factoids

References

Would you like to implement a fix?

github-actions bot commented Jun 14, 2023

Community Note

tom-dalton-fanduel commented Jun 20, 2023 • edited Loading

wad-hongsumin commented Jun 24, 2023

oarmstrong commented Jun 30, 2023

oarmstrong commented Jun 30, 2023 • edited Loading

oarmstrong commented Jun 30, 2023 • edited Loading

johnsonaj commented Jun 30, 2023

oarmstrong commented Jul 1, 2023

ghost commented Jul 12, 2023

Prashanteppalpalli commented Aug 21, 2023 • edited Loading

tonyfosterdev commented Oct 7, 2023 • edited Loading

boreasoss commented Nov 21, 2023

mikeizzy commented Dec 17, 2023

danielw-aws commented Jan 4, 2024

elyabesantos commented Feb 2, 2024

github-actions bot commented Mar 7, 2024

github-actions bot commented Apr 7, 2024

suminhong commented Jun 14, 2023 •

edited

Loading

tom-dalton-fanduel commented Jun 20, 2023 •

edited

Loading

oarmstrong commented Jun 30, 2023 •

edited

Loading

oarmstrong commented Jun 30, 2023 •

edited

Loading

Prashanteppalpalli commented Aug 21, 2023 •

edited

Loading

tonyfosterdev commented Oct 7, 2023 •

edited

Loading