Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add attribute mv and replace commands #111

Merged
merged 3 commits into from
Feb 12, 2025
Merged

Add attribute mv and replace commands #111

merged 3 commits into from
Feb 12, 2025

Conversation

minamijoyo
Copy link
Owner

@minamijoyo minamijoyo commented Feb 5, 2025

Summary

This pull request adds attribute mv and attribute replace commands.

Why

When working on dependency upgrades, I often come across warnings such as "A is deprecated. Use B instead". The current functionality only allows removing the old attribute and appending a new one to the end, which changes the order of the attributes and loses valuable comments. It would be great if we could rename an attribute.

For example, Terraform v1.10 introduced DynamoDB-free S3-native state locking in the s3 backend, and the upcoming Terraform v1.11 will start deprecating the old dynamodb_table attribute and recommend using the new use_lockfile. hashicorp/terraform#36257
It's easy to rewrite a few files by hand, but it could be tedious if you have hundreds of backend configurations across many repositories.

What

This pull request adds attribute mv and attribute replace commands to help such an upgrade work.

Note: As for implementation, hclwrite's attribute name is currently a private variable with no API for renaming it. Therefore, a forked version of hclwrite is used to implement this feature. So, we need to wait for the upstream patch to be merged before we merge this implementation. The upstream patch can be found at hashicorp/hcl#506. (Edit: The upstream patch has been merged 🎉 )

Regarding the design, we have aligned the attribute mv command with block mv. It accepts from and to addresses so that it can be extended in the future, but we have not implemented moving an attribute across blocks yet.

$ hcledit attribute mv --help
Move attribute (Rename attribute key)

Arguments:
  FROM_ADDRESS     An old address of attribute.
  TO_ADDRESS       A new address of attribute.

Usage:
  hcledit attribute mv <FROM_ADDRESS> <TO_ADDRESS> [flags]

Flags:
  -h, --help   help for mv

Global Flags:
  -f, --file string   A path of input file (default "-")
  -u, --update        Update files in-place

In addition to the attribute mv command, this PR also adds a new attribute replace command. This command not only renames the key of an attribute but also sets the value of the attribute. This could be done by combining attribute mv and attribute set, but doing it with a single command would be more convenient. This is because in dependency upgrade scenarios, where attribute A is deprecated and use B instead, it is often the case that not only the name of the attribute changes but also its value.

$ hcledit attribute replace --help
Replace both the name and value of matched attribute at a given address

Arguments:
  ADDRESS          An address of attribute to be replaced.
  NAME             A new name (key) of attribute.
  VALUE            A new value of attribute.
                   The value is set literally, even if references or expressions.
                   Thus, if you want to set a string literal "bar", be sure to
                   escape double quotes so that they are not discarded by your shell.
                   e.g.) hcledit attribute replace aaa.bbb.ccc foo '"bar"'

Usage:
  hcledit attribute replace <ADDRESS> <NAME> <VALUE> [flags]

Flags:
  -h, --help   help for replace

Global Flags:
  -f, --file string   A path of input file (default "-")
  -u, --update        Update files in-place

Examples

$ cat tmp/attr.hcl
resource "foo" "bar" {
  attr1 = "val1"
  nested {
    attr2 = "val2"
  }
}
$ cat tmp/attr.hcl | hcledit attribute mv resource.foo.bar.nested.attr2 resource.foo.bar.nested.attr3
resource "foo" "bar" {
  attr1 = "val1"
  nested {
    attr3 = "val2"
  }
}
$ cat tmp/attr.hcl | hcledit attribute replace resource.foo.bar.nested.attr2 attr3 '"val3"'
resource "foo" "bar" {
  attr1 = "val1"
  nested {
    attr3 = "val3"
  }
}

An example of rewriting s3 backend dynamodb_table to use_lockfile in Terraform v1.11 upgrade:

$ cat tmp/s3lock/main.tf
terraform {
  backend "s3" {
    region         = "ap-northeast-1"
    bucket         = "my-s3lock-test"
    key            = "dir1/terraform.tfstate"
    dynamodb_table = "tflock"
    profile        = "foo"
  }
}
$ cat tmp/s3lock/main.tf | hcledit attribute replace terraform.backend.s3.dynamodb_table use_lockfile true
terraform {
  backend "s3" {
    region       = "ap-northeast-1"
    bucket       = "my-s3lock-test"
    key          = "dir1/terraform.tfstate"
    use_lockfile = true
    profile      = "foo"
  }
}

You can combine it with find, xargs, etc., and rewrite multiple files at once:

$ find . -name "*.tf" | xargs -I{} hcledit attribute replace terraform.backend.s3.dynamodb_table use_lockfile true -f {} -u

While my initial motivation was the above, I'm sure there are many more real-world use cases, not just this one.

@minamijoyo minamijoyo changed the title Add attribute mv (rename) and replace commands Add attribute mv and replace commands Feb 5, 2025
I believe the pattern "attribute A is deprecated, use B instead” is a
generic use case. The current functionality only allows removing the old
attribute and appending a new one to the end, which changes the order of
the attributes. I don't want to change the order because meaningful
order contributes to reducing cognitive load. To help with such a
transition, it would be great if we could rename an attribute.

Regarding the design, we have aligned the attribute mv command with
block mv but have not implemented moving across blocks yet. However, it
accepts an address as an argument so that it can be extended in the
future.

As for implementation, hclwrite's attribute name is currently a private
variable with no API for renaming it. Therefore, we will use the forked
version until the patch is merged.
In addition to the attribute mv command, add a new attribute replace
command. This command not only renames the key of an attribute but also
sets the value of the attribute. This could be done by combining
attribute mv and attribute set, but doing it with a single command would
be more convenient. This is because in dependency upgrade scenarios,
where attribute A is deprecated and use B instead, it is often the case
that not only the name of the attribute changes but also its value.

For example, Terraform v1.10 introduced DynamoDB-free S3-native state
locking in the s3 backend, and the upcoming Terraform v1.11 will start
deprecating the old dynamodb_table attribute and recommend using the new
use_lockfile. It's easy to rewrite a few files by hand, but it could be
tedious if you have hundreds of backend configurations across many
repositories.

While the above is the initial motivation for implementing this feature,
it is not hard to imagine that the real-world use cases go beyond this.

As with the attribute mv command, a forked version of hclwrite is used
to implement this feature. So, we need to wait for the upstream patch to
be merged before we merge this implementation.
The patch for upstream needed to implement the attribute mv/replace
command has been merged.
hashicorp/hcl#506

So use the latest main branch of upstream instead of the forked version.
@minamijoyo minamijoyo merged commit 80d76cc into master Feb 12, 2025
4 checks passed
@minamijoyo minamijoyo deleted the attribute-replace branch February 12, 2025 01:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant