TypeSet with nested TypeMap shows empty map in plan output when a map is removed from config

[anGie44]

SDK version

"v2.0.1" (also tested against v1)

Relevant provider source code

Excerpt from aws/resource_aws_autoscaling_group.go (v3.7.0):

"tags": {
				Type:     schema.TypeSet,
				Optional: true,
				Elem: &schema.Schema{
					Type: schema.TypeMap,
					Elem: &schema.Schema{Type: schema.TypeString},
				},

Terraform Configuration Files

https://gist.github.com/anGie44/417856ad8231c96a89cf9e0d0356817f

Debug Output

020/09/20 02:02:43 [WARN] Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for aws_autoscaling_group.bar, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .tags: planned value cty.SetVal([]cty.Value{cty.MapVal(map[string]cty.Value{"key":cty.StringVal("Name"), "propagate_at_launch":cty.StringVal("true"), "value":cty.StringVal("example")}), cty.MapValEmpty(cty.String)}) does not match config value cty.SetVal([]cty.Value{cty.MapVal(map[string]cty.Value{"key":cty.StringVal("Name"), "propagate_at_launch":cty.StringVal("true"), "value":cty.StringVal("example")})}) nor prior value cty.SetVal([]cty.Value{cty.MapVal(map[string]cty.Value{"key":cty.StringVal("Name"), "propagate_at_launch":cty.StringVal("true"), "value":cty.StringVal("example")}), cty.MapVal(map[string]cty.Value{"key":cty.StringVal("Owner"), "propagate_at_launch":cty.StringVal("true"), "value":cty.StringVal("bezos")})})

Expected Behavior

   ~ tags                      = [
            {
                "key"                 = "Name"
                "propagate_at_launch" = "true"
                "value"               = "example"
            },
          - {
              - "key"                 = "Owner"
              - "propagate_at_launch" = "true"
              - "value"               = "bezos"
            },
        ]

Actual Behavior

    ~ tags                      = [
            {
                "key"                 = "Name"
                "propagate_at_launch" = "true"
                "value"               = "example"
            },
          - {
              - "key"                 = "Owner"
              - "propagate_at_launch" = "true"
              - "value"               = "bezos"
            },
          + {},
        ]

Steps to Reproduce

1. Using main.tf:
   a. terraform init
   b. terraform apply

2. Using update.tf (same as main.tf but with 2nd map in tags removed):
   a. terraform plan

References

• Unwarranted changes when updating tags on aws_autoscaling_group terraform-provider-aws#15227

[anGie44]

Also reproducible with test in field_reader_diff_test.go:

func TestDiffFieldReader_MapHandling(t *testing.T) {
	schema := map[string]*Schema{
		"tags": {
			Type:    TypeSet,
			Optional: true,
			Elem: &Schema{
				Type: TypeMap,
				Elem: &Schema{Type: TypeString},
			},
		},
	}
	r := &DiffFieldReader{
		Schema: schema,
		Diff: &terraform.InstanceDiff{
			Attributes: map[string]*terraform.ResourceAttrDiff{
				"tags.#": {
					Old: "2",
					New: "1",
				},
				"tags.1.key": {
					Old: "Owner",
					New: "",
					NewRemoved: true,
				},
				"tags.1.value": {
					Old: "gates",
					New: "",
					NewRemoved: true,
				},
				"tags.1.propagate_at_launch": {
					Old: "true",
					New: "",
					NewRemoved: true,
				},
			},
		},
		Source: &MapFieldReader{
			Schema: schema,
			Map: BasicMapReader(map[string]string{
				"tags.#":   "2",
				"tags.0.key": "Name",
				"tags.0.value": "example",
				"tags.0.propagate_at_launch": "true",
				"tags.1.key": "Owner",
				"tags.1.value": "gates",
				"tags.1.propagate_at_launch": "true",
			}),
		},
	}

	result, err := r.ReadField([]string{"tags"})
	if err != nil {
		t.Fatalf("ReadField failed: %#v", err)
	}
	expected := NewSet(func(i interface{}) int {
		var buf bytes.Buffer
		for _, v := range i.(map[string]interface{}) {
			buf.WriteString(fmt.Sprintf("%s-", v.(string)))
		}
		return HashString(buf.String())
	}, []interface{}{
		map[string]interface{}{"key": "Name", "value": "example", "propagate_at_launch": "true"},
	})

	if !reflect.DeepEqual(expected, result.Value) {
		t.Fatalf("bad: DiffHandling\n\nexpected: %#v\n\ngot: %#v\n\n", expected, result.Value)
	}
}`

Output:

=== RUN   TestDiffFieldReader_MapHandling
    TestDiffFieldReader_MapHandling: field_reader_diff_test.go:180: bad: DiffHandling
        
        expected: *Set(map[string]interface {}{"1482494236":map[string]interface {}{"key":"Name", "propagate_at_launch":"true", "value":"example"}})
        
        got: *Set(map[string]interface {}(nil))
        
--- FAIL: TestDiffFieldReader_MapHandling (0.00s)
FAIL

[bflad]

This behavior also happens with deeply nested TypeMap under TypeSet, e.g. this schema:

			"distribution": {
				Type:     schema.TypeSet,
				Required: true,
				Elem: &schema.Resource{
					Schema: map[string]*schema.Schema{
						"ami_distribution_configuration": {
							Type:     schema.TypeList,
							Optional: true,
							MaxItems: 1,
							Elem: &schema.Resource{
								Schema: map[string]*schema.Schema{
									"ami_tags": {
										Type:     schema.TypeMap,
										Optional: true,
										Elem:     &schema.Schema{Type: schema.TypeString},
									},
// ...
								},
							},
						},
// ...
					},
				},
			},

Will generate an empty distribution element on update of the nested ami_tags attribute.