[Enhancement]: EventBridge Pipes

[bradwood]

Description

Please add support for EventBridge Pipes -- see https://aws.amazon.com/eventbridge/pipes/

Affected Resource(s) and/or Data Source(s)

No response

Potential Terraform Configuration

No response

References

No response

Would you like to implement a fix?

No

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[roberth-k]

I've picked up aws_pipes_pipe. It's quite a large resource, so it may take a week or two before it's complete.

[ad-m-ss]

FTR: Anybody interested to give a shot to AWS EventBridge Pipes may use temporary aws_cloudformation_stack and https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-pipes-pipe.html as a workaround:

resource "aws_cloudformation_stack" "cloudformation_stack" {
  name = "CostAnomalyUpdates"

  parameters = {
    RoleArn   = aws_iam_role.cost_anomaly_updates_event_bridges_role.arn
    SourceArn = aws_sqs_queue.cost_anomaly_updates.arn
    TargetArn = aws_cloudwatch_log_group.cloudwatch-group.arn
  }

  template_body = jsonencode({
    "Parameters" : {
      "SourceArn" : {
        "Type" : "String",
      },
      "TargetArn" : {
        "Type" : "String",
      },
      "RoleArn" : {
        "Type" : "String"
      }
    },
    "Resources" : {
      "CopyPipe" : {
        "Type" : "AWS::Pipes::Pipe",
        "Properties" : {
          "Name" : "CostAnomalyUpdates-SQS-to-CW",
          "RoleArn" : { "Ref" : "RoleArn" }
          "Source" : { "Ref" : "SourceArn" },
          "Target" : { "Ref" : "TargetArn" },
        }
      }
    }
  })
}

[roberth-k]

Apologies, this is still in progress... Some of the sources and targets have invisible, un-documented default parameter values, which has been difficult to reconcile with Terraform's diffs under some combinations of resource updates. I'm slowly inching towards solving the edge cases, but it may yet be some time before this is complete.

[gianlo]

The best docs I found are on the CLI's documentation:
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/pipes/create-pipe.html
@roberth-k

[SolomonHD]

@roberth-k Your changes were merged into main a couple of months ago; did this make it to a new terraform version?

ETA: I see the pipes module is part of awscc provider, not aws as I was expecting.

May I have an example for creating a Pipe with an active_mq source and bus target? I don't see where to put the ARN of the active mq, I only see the basic_auth and queue name parameters.

[brittandeyoung]

Looks like this issue did not get closed with the PR merge. Closing it out as the resource in the pr covers the request.

[brittandeyoung]

@roberth-k was this left open intentionally? I want to make sure I am only closing the issue if it’s actually completed.

[roberth-k]

Hi @brittandeyoung -- #30538 is a partial implementation, so this issue should remain open.

[jordanfinners]

Working with @roberth-k on filling out the parameters etc :)
There's an awful lot of sources and targets, so bear with us!

[jordanfinners]

Working through the tests now! Hopefully shouldn't be too much longer :)

[jordanfinners]

I've got a PR open, if you give it thumbs up, it'll help get prioritised #31607 (comment)

[github-actions]

This functionality has been released in v5.6.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.