Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provider incorrectly seems to detect a change, applies it - next apply incorrectly detects the same change #4171

Closed
data-henrik opened this issue Nov 15, 2022 · 7 comments · Fixed by #5246
Labels
service/CBR Issues related to Context Based Restrictions

Comments

@data-henrik
Copy link
Contributor

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform IBM Provider Version

$ terraform -v
Terraform v1.3.4
on linux_amd64
+ provider registry.terraform.io/hashicorp/kubernetes v2.11.0
+ provider registry.terraform.io/hashicorp/null v3.2.0
+ provider registry.terraform.io/ibm-cloud/ibm v1.47.1

Affected Resource(s)

  • ibm_cbr_rule

Terraform Configuration Files

This is the resource which I could apply n times, like in an infinite loop.

# registry access
resource "ibm_cbr_rule" "cbr_rule_registry" {
  contexts {
    attributes {
      name  = "networkZoneId"
      value = ibm_cbr_zone.cbr_zone_k8s.id
    }
  }

  description      = "restrict access to registry to cluster"
  enforcement_mode = var.cbr_enforcement_mode
  resources {
    attributes {
      name  = "accountId"
      value = data.ibm_iam_account_settings.team_iam_account_settings.account_id
    }
    attributes {
      name     = "resourceType"
      operator = "stringEquals"
      value    = "namespace"
    }
    attributes {
      name     = "resource"
      operator = "stringEquals"
      value    = "henrik/bluegreen"
    }
    attributes {
      name     = "serviceName"
      operator = "stringEquals"
      value    = "container-registry"
    }

  }
}

Debug Output```

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
~ update in-place

Terraform will perform the following actions:

ibm_cbr_rule.cbr_rule_registry will be updated in-place

~ resource "ibm_cbr_rule" "cbr_rule_registry" {
id = "31e9546169c70af51a0899ad8d7d4e8d"
# (9 unchanged attributes hidden)

  ~ resources {
      ~ attributes {
          ~ name     = "resource" -> "resourceType"
          ~ value    = "henrik/bluegreen" -> "namespace"
            # (1 unchanged attribute hidden)
        }
      ~ attributes {
          ~ name     = "resourceType" -> "resource"
          ~ value    = "namespace" -> "henrik/bluegreen"
            # (1 unchanged attribute hidden)
        }

        # (2 unchanged blocks hidden)
    }

    # (1 unchanged block hidden)
}

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.

Enter a value: yes

ibm_cbr_rule.cbr_rule_registry: Modifying... [id=31e9546169c70af51a0899ad8d7d4e8d]
ibm_cbr_rule.cbr_rule_registry: Modifications complete after 1s [id=31e9546169c70af51a0899ad8d7d4e8d]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

<!---
Please provide a link to a GitHub Gist containing the complete debug output. Please do NOT paste the debug output in the issue; just paste a link to the Gist.

To obtain the debug output, see the [Terraform documentation on debugging](https://www.terraform.io/docs/internals/debugging.html).
--->

### Panic Output

<!--- If Terraform produced a panic, please provide a link to a GitHub Gist containing the output of the `crash.log`. --->

### Expected Behavior
I expect the resource, when applied once, to be stable. When I call "apply" again there should be not changes.
<!--- What should have happened? --->

### Actual Behavior
Whenever I apply again, a change is detected and the resource updated. The ID remains the same. On the next apply, the same resource is updated again.
<!--- What actually happened? --->

### Steps to Reproduce

<!--- Please list the steps required to reproduce the issue. --->

1. `terraform apply`

### Important Factoids

<!--- Are there anything atypical about your accounts that we should know? For example: Running in EC2 Classic? --->

### References

<!---
Information about referencing Github Issues: https://help.github.com/articles/basic-writing-and-formatting-syntax/#referencing-issues-and-pull-requests

Are there any other GitHub issues (open or closed) or pull requests that should be linked here? Vendor documentation? For example:
--->

* #0000
@github-actions github-actions bot added the service/CBR Issues related to Context Based Restrictions label Nov 15, 2022
@daniel-butler-irl
Copy link

I am also seeing the same issue and it creates problems for me since I have tests to ensure my modules are idempotent. This breaks me. Here is a full end-to-end example

terraform {
  required_version = ">= 1.0.0"
  required_providers {
    # Pin to the lowest provider version of the range defined in the main module to ensure lowest version still works
    ibm = {
      source  = "ibm-cloud/ibm"
      version = "1.45.0"
    }
  }
}

variable "ibmcloud_api_key" {
  type        = string
  description = "The IBM Cloud API Token"
  sensitive   = true
}

provider "ibm" {
  ibmcloud_api_key = var.ibmcloud_api_key
}

resource "ibm_resource_group" "resourceGroup" {
  name     = "test-rg-cbr"
}

resource "ibm_is_vpc" "example_vpc" {
  name           = "test-vpc-cbr"
  resource_group = ibm_resource_group.resourceGroup.id
}

resource "ibm_is_public_gateway" "testacc_gateway" {
  name           = "test-pgateway-cbr"
  vpc            = ibm_is_vpc.example_vpc.id
  zone           = "us-south-1"
  resource_group = ibm_resource_group.resourceGroup.id
}

resource "ibm_is_subnet" "testacc_subnet" {
  name                     = "test-subnet-cbr"
  vpc                      = ibm_is_vpc.example_vpc.id
  zone                     = "us-south-1"
  public_gateway           = ibm_is_public_gateway.testacc_gateway.id
  total_ipv4_address_count = 256
  resource_group           = ibm_resource_group.resourceGroup.id
}

resource "ibm_database" "redis_database" {
  resource_group_id                    = ibm_resource_group.resourceGroup.id
  name                                 = "test-redis-cbr"
  service                              = "databases-for-redis"
  plan                                 = "standard"
  location                             = "us-south"
}

data "ibm_iam_account_settings" "iam_account_settings" {
}

resource "ibm_cbr_zone" "cbr_zone" {
  account_id = data.ibm_iam_account_settings.iam_account_settings.id
  addresses {
    type  = "vpc" # to bind a specific vpc to the zone
    value = ibm_is_vpc.example_vpc.crn
  }
  description = "this is an example of zone"

  name = "test-redis-zone"
}

resource "ibm_cbr_rule" "cbr_rule" {
  contexts {
    attributes {
      name  = "networkZoneId"
      value = ibm_cbr_zone.cbr_zone.id
    }
  }
  description = "this is an example of rule"
  enforcement_mode = "enabled"
  operations {
    api_types {
      api_type_id = "crn:v1:bluemix:public:context-based-restrictions::::api-type:data-plane"
    }
  }
  resources {
    attributes {
      name     = "accountId"
      value    = data.ibm_iam_account_settings.iam_account_settings.id
      operator = "stringEquals"
    }
    attributes {
      name     = "serviceName"
      value    = "databases-for-redis"
      operator = "stringEquals"
    }
    attributes {
      name     = "serviceInstance"
      value    = ibm_database.redis_database.id
      operator = "stringEquals"
    }

  }
}

Every time after applying if we do a plan changes are detected. The change is the order in which the attributes return is different every time. Even if we apply the change next time a change is detected. Here is the output:

terraform plan
data.ibm_iam_account_settings.iam_account_settings: Reading...                                                                       
ibm_resource_group.resourceGroup: Refreshing state... [id=e9957c89eed5494bb717c73a0ddcca8d]
data.ibm_iam_account_settings.iam_account_settings: Read complete after 0s [id=abac0df06b644a9cabc6e44f55b3880e]
ibm_is_vpc.example_vpc: Refreshing state... [id=r006-577ceb53-30b3-4b01-b981-3fb6ea39d157]
ibm_database.redis_database: Refreshing state... [id=crn:v1:bluemix:public:databases-for-redis:us-south:a/abac0df06b644a9cabc6e44f55b3880e:3990ce80-133e-4d05-8bb8-36c608dbe144::]
ibm_is_public_gateway.testacc_gateway: Refreshing state... [id=r006-26536c02-cb1f-4d42-8b51-07c120713f4b]
ibm_cbr_zone.cbr_zone: Refreshing state... [id=366d56825bd272e7e7ffe0299e10581c]
ibm_cbr_rule.cbr_rule: Refreshing state... [id=4fbe9ab9d9277c48b9a024aa0a1a7dca]
ibm_is_subnet.testacc_subnet: Refreshing state... [id=0717-c99e549c-68c6-477a-ad9b-935f8c8cdd60]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
  ~ update in-place

Terraform will perform the following actions:

  # ibm_cbr_rule.cbr_rule will be updated in-place
  ~ resource "ibm_cbr_rule" "cbr_rule" {
        id                  = "4fbe9ab9d9277c48b9a024aa0a1a7dca"
        # (9 unchanged attributes hidden)

      ~ resources {
          ~ attributes {
              ~ name     = "serviceInstance" -> "serviceName"
              ~ value    = "crn:v1:bluemix:public:databases-for-redis:us-south:a/abac0df06b644a9cabc6e44f55b3880e:3990ce80-133e-4d05-8bb8-36c608dbe144::" -> "databases-for-redis"
                # (1 unchanged attribute hidden)
            }
          ~ attributes {
              ~ name     = "serviceName" -> "serviceInstance"
              ~ value    = "databases-for-redis" -> "crn:v1:bluemix:public:databases-for-redis:us-south:a/abac0df06b644a9cabc6e44f55b3880e:3990ce80-133e-4d05-8bb8-36c608dbe144::"
                # (1 unchanged attribute hidden)
            }

            # (1 unchanged block hidden)
        }

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

@ocofaigh
Copy link
Contributor

It may be the same root cause, but I am seeing the same issue with ibm_schematics_workspace resource.
After initial terraform apply has completed, running a subsequent apply, terraform is picking up an update operation, however there is no actual update. The order of the value for template_env_settings is being detected as an update, even though values are the same, but just in a different order:

Before:

{"template_env_settings":[{"__netrc__":"[[\"github.ibm.com\",\"username\",\"git-token\"]]"},{"API_DATA_IS_SENSITIVE":"true"}]}

After:

{"template_env_settings":[{"API_DATA_IS_SENSITIVE":"true"},{"__netrc__":"[[\"github.ibm.com\",\"username\",\"git-token\"]]"}]}`

We have tests to check that a module is idempotent, and this bug is failing our tests.

@hkantare
Copy link
Collaborator

hkantare commented Feb 1, 2023

@zhenwan Can you please look into the issue

@zhenwan
Copy link
Contributor

zhenwan commented Feb 2, 2023

@data-henrik The order of attributes block in your resource config is different from the one returned from CBR service. you can find out the order of attributes returned from CBR service in the terraform.tfstate file.

To prevent this behavior from happening, you can change order of the attributes to be the same as terraform.tfstate file:

# registry access
resource "ibm_cbr_rule" "cbr_rule_registry" {
  contexts {
    attributes {
      name  = "networkZoneId"
      value = ibm_cbr_zone.cbr_zone_k8s.id
    }
  }

  description      = "restrict access to registry to cluster"
  enforcement_mode = var.cbr_enforcement_mode
  resources {
    attributes {
      name  = "accountId"
      value = data.ibm_iam_account_settings.team_iam_account_settings.account_id
    }
    attributes {
      name     = "resource"
      operator = "stringEquals"
      value    = "henrik/bluegreen"
    }
    attributes {
      name     = "resourceType"
      operator = "stringEquals"
      value    = "namespace"
    }
    attributes {
      name     = "serviceName"
      operator = "stringEquals"
      value    = "container-registry"
    }

  }
}

@data-henrik
Copy link
Contributor Author

That might be a workaround to cope with this bug. But it is not a solution. The order should not matter.

@hkantare
Copy link
Collaborator

hkantare commented Feb 2, 2023

@zhenwan That a workaround may can you look into changing the Type from list to set . As per Terraform development we should Type set in resources instead of list because of the order issue.

@zhenwan
Copy link
Contributor

zhenwan commented Feb 2, 2023

@hkantare it sounds like this is a known issue. Do you know if this issue has been fixed in the Terraform code generator? Do you have a guideline / doc to fix this order issue ? From CBR service API standpoint, we don't care what the order is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
service/CBR Issues related to Context Based Restrictions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants