Add separated resource for configuring S3 bucket lifecycle rules

[hashibot]

This issue was originally opened by @pabloajz as hashicorp/terraform#8961. It was migrated here as part of the provider split. The original body of the issue is below.

---

Add a separate resource aws_s3_bucket_lifecycle_rule for supporting lifecycle rules (http://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html)

Current situation

Lifecycle rules can be defined only inline in the aws_s3_bucket resource, but not as external resources than can be attached to the S3 bucket afterwards

Use Case

I have one S3 bucket containing Logs organised into different keys. Each key is generated by other components (different Log Groups for example). I want to setup different lifecycle rules for each Log Group (Key) when they are created, and they are created dynamically after the S3 bucket is created.

[igor-khomyshyn-clarivate]

+1

[jasonkuehl]

+1

[ringods]

Please stop using the +1 as additional comments, but add the thumbs up emoji to the original issue description.

[devinsba]

I've started on an impl for this feature but per the discussion in: #916 it seems that this resource would most likely not get merged in

Thoughts on whether I should continue?

[jayanderson]

@devinsba It looks like #916 was eventually merged. I'd say finish it up if you were close. This is a desired feature by many. It'd be nice to get the terraform developers to chime in on whether it would be a welcome pull request before committing too much time.

[devinsba]

I stopped immediately when I posted that. I don't know if that code still exists, someone else should take a crack though

[zeeshanabid94]

+1

[kvz]

Not sure if this is related enough, but I have the case where there's one leading bucket that I want to replicate to a follower bucket for every each new region that we expand to. Ideally, the replication is hence part of the follower hcl, and replication rules can be attached to the leading bucket from there.

[eedwards-sk]

Any updates on this? Numerous +1s over the past 2 years, yet still no solution I've found.
I was about to open a new issue on this, so instead I'll paste it below:

Current Terraform Version

0.11.10

Use-cases

Create and add lifecycle policies to an s3 bucket that was not created by terraform.

A common use case is a project that needs a bucket and stores files in it, but the bucket is shared by multiple terraform projects where none of them own the bucket.

This configuration prevents any individual project from destroying the bucket, but currently does not allow them to all manage their own necessary lifecycle policies for object deletion.

e.g. if I have a bucket

my-projects

and inside I have 2 prefixes, foo and bar, denoting where some files are generated for each one:

foo/things/*.tar.gz
bar/stuff/*.zip

then I want two separate terraform projects to be able to create independent lifecycle policy rules for their own appropriate namespace

e.g. the foo project would create a lifecycle policy for foo/things/*.tar.gz

Attempted Solutions

No solution available. I would have to create a separate bucket for each terraform project. Not ideal.

Proposal

allow lifecycle_rule configuration as a separate top-level resource (e.g. ripped from the aws_s3_bucket resource):

resource "aws_s3_bucket_lifecycle_rule" "my_rule" {
  bucket = "my-bucket"
  id      = "log"
  enabled = true

  prefix = "log/"

  tags {
    "rule"      = "log"
    "autoclean" = "true"
  }

  transition {
    days          = 30
    storage_class = "STANDARD_IA" # or "ONEZONE_IA"
  }

  transition {
    days          = 60
    storage_class = "GLACIER"
  }

  expiration {
    days = 90
  }
}

References

• aws_iam_policy_document should combine redundant actions #10770

[ringods]

@eedwards-sk I wouldn't say that Hashicorp is ignorant on this one. If you followed the Terraform 0.12 blog posts, you might have bumped into the upcoming Dynamic Blocks support. Given this low-level syntax support, there is less the need to extract some of these nested blocks as top level resources. We just have to be a tad more patient. 😉

[eedwards-sk]

@ringods while that feature looks useful for DRYing up templates, unless I'm missing something, it unfortunately does nothing for solving this issue

This issue is for the need for a top-level resource to create bucket lifecycle rules. I'm not looking to make it easier to repeat similar rules across multiple bucket resources, but to be able to create rules for a bucket that the terraform project did not create.

[fizban79]

Is there any movement on this request? Or it makes sense to wait 0.12 with dynamic nested blocks?

[eedwards-sk]

@fizban79 as I commented above, dynamic nested blocks do not solve the issue

[fizban79]

@eedwards-sk agreed, they do not solve all cases of the issue. But their functionality is enough to solve my problems. :)

[github-actions]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

[jayanderson]

Commenting to keep this open. It'd be fine to merge this with #4418 if that's the decision.

[github-actions]

This functionality has been released in v4.0.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.