Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: sns topic creation should not be idempotent #3798

Closed
wants to merge 6 commits into from
Closed

fix: sns topic creation should not be idempotent #3798

wants to merge 6 commits into from

Conversation

corymhall
Copy link
Contributor

@corymhall corymhall commented Apr 10, 2024 •
edited
Loading

This is a re-roll of #3235.

See this comment for why we are re-rolling this.

closes #3288

Co-authored-by: VenelinMartinov [email protected]

@corymhall @VenelinMartinov
fix: sns topic creation should not be idempotent
e8783ce 
This is a re-roll of #3235.

See [this comment](#3288 (comment))
for why we are re-rolling this.

closes #3288

Co-authored-by: VenelinMartinov <[email protected]>
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 10, 2024 •
edited
Loading

Does the PR have any schema changes?

Does the PR have any schema changes?

Found 33 breaking changes:

Resources

  • "aws:appstream/fleet:Fleet":
    • 🟡 inputs: "maxSessionsPerInstance" missing
    • 🟡 properties: "maxSessionsPerInstance" missing output "maxSessionsPerInstance"
  • "aws:batch/jobDefinition:JobDefinition":
    • 🟡 inputs: "deregisterOnNewRevision" missing
    • 🟡 properties: "deregisterOnNewRevision" missing output "deregisterOnNewRevision"
  • 🔴 "aws:devopsguru/serviceIntegration:ServiceIntegration" missing
  • "aws:emr/cluster:Cluster":
    • 🟡 inputs: "unhealthyNodeReplacement" missing
    • 🟡 properties: "unhealthyNodeReplacement" missing output "unhealthyNodeReplacement"
  • 🟡 "aws:servicecatalog/appregistryApplication:AppregistryApplication": properties: "applicationTag" missing output "applicationTag"
  • "aws:transfer/server:Server":
    • 🟡 inputs: "s3StorageOptions" missing
    • 🟡 properties: "s3StorageOptions" missing output "s3StorageOptions"

Functions

  • 🔴 "aws:devopsguru/getNotificationChannel:getNotificationChannel" missing
  • 🔴 "aws:devopsguru/getResourceCollection:getResourceCollection" missing
  • 🔴 "aws:ecr/getLifecyclePolicyDocument:getLifecyclePolicyDocument" missing

Types

  • "aws:appstream/FleetComputeCapacity:FleetComputeCapacity":
    • 🟡 properties: "desiredSessions" missing
    • 🟢 required: "desiredInstances" property has changed to Required
  • 🔴 "aws:devopsguru/ServiceIntegrationKmsServerSideEncryption:ServiceIntegrationKmsServerSideEncryption" missing
  • 🔴 "aws:devopsguru/ServiceIntegrationLogsAnomalyDetection:ServiceIntegrationLogsAnomalyDetection" missing
  • 🔴 "aws:devopsguru/ServiceIntegrationOpsCenter:ServiceIntegrationOpsCenter" missing
  • 🔴 "aws:devopsguru/getNotificationChannelFilter:getNotificationChannelFilter" missing
  • 🔴 "aws:devopsguru/getNotificationChannelSn:getNotificationChannelSn" missing
  • 🔴 "aws:devopsguru/getResourceCollectionCloudformation:getResourceCollectionCloudformation" missing
  • 🔴 "aws:devopsguru/getResourceCollectionTag:getResourceCollectionTag" missing
  • 🔴 "aws:ecr/getLifecyclePolicyDocumentRule:getLifecyclePolicyDocumentRule" missing
  • 🔴 "aws:ecr/getLifecyclePolicyDocumentRuleAction:getLifecyclePolicyDocumentRuleAction" missing
  • 🔴 "aws:ecr/getLifecyclePolicyDocumentRuleSelection:getLifecyclePolicyDocumentRuleSelection" missing
  • 🟡 "aws:glue/DataCatalogEncryptionSettingsDataCatalogEncryptionSettingsEncryptionAtRest:DataCatalogEncryptionSettingsDataCatalogEncryptionSettingsEncryptionAtRest": properties: "catalogEncryptionServiceRole" missing
  • "aws:glue/getDataCatalogEncryptionSettingsDataCatalogEncryptionSettingEncryptionAtRest:getDataCatalogEncryptionSettingsDataCatalogEncryptionSettingEncryptionAtRest":
    • 🟡 properties: "catalogEncryptionServiceRole" missing
    • 🟢 required: "catalogEncryptionServiceRole" property is no longer Required
  • 🔴 "aws:transfer/ServerS3StorageOptions:ServerS3StorageOptions" missing
  • "aws:wafv2/WebAclRuleStatementManagedRuleGroupStatementManagedRuleGroupConfigAwsManagedRulesAcfpRuleSetRequestInspection:WebAclRuleStatementManagedRuleGroupStatementManagedRuleGroupConfigAwsManagedRulesAcfpRuleSetRequestInspection": properties:
    • 🟡 "addressFields" missing
    • 🟡 "phoneNumberFields" missing
  • 🔴 "aws:wafv2/WebAclRuleStatementManagedRuleGroupStatementManagedRuleGroupConfigAwsManagedRulesAcfpRuleSetRequestInspectionAddressFields:WebAclRuleStatementManagedRuleGroupStatementManagedRuleGroupConfigAwsManagedRulesAcfpRuleSetRequestInspectionAddressFields" missing
  • 🔴 "aws:wafv2/WebAclRuleStatementManagedRuleGroupStatementManagedRuleGroupConfigAwsManagedRulesAcfpRuleSetRequestInspectionPhoneNumberFields:WebAclRuleStatementManagedRuleGroupStatementManagedRuleGroupConfigAwsManagedRulesAcfpRuleSetRequestInspectionPhoneNumberFields" missing
    No new resources/functions.

Maintainer note: consult the runbook for dealing with any breaking changes.

@github-actions GitHub Actions
Copy link

Is README.md missing any configuration options?

assumeRoleWithWebIdentity not found in Configuration section
customCaBundle not found in Configuration section
defaultTags not found in Configuration section
ec2MetadataServiceEndpoint not found in Configuration section
ec2MetadataServiceEndpointMode not found in Configuration section
httpProxy not found in Configuration section
httpsProxy not found in Configuration section
ignoreTags not found in Configuration section
noProxy not found in Configuration section
retryMode not found in Configuration section
s3UsEast1RegionalEndpoint not found in Configuration section
sharedConfigFiles not found in Configuration section
skipRequestingAccountId not found in Configuration section
stsRegion not found in Configuration section
tokenBucketRateLimiterCapacity not found in Configuration section
useDualstackEndpoint not found in Configuration section

Please add a description for each of these options to README.md. Details about them can be found in either the upstream docs or schema.json.

@corymhall corymhall requested a review from t0yv0 April 10, 2024 18:47
t0yv0
t0yv0 reviewed Apr 10, 2024
View reviewed changes
patches/0049-non-idempotent-sns-topic-creation.patch Outdated
+ return fmt.Sprintf("arn:aws:sns:%s:%s:%s", region, account, snsTopicName)
+}
+
+var snsGlobalMutex sync.Mutex
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know I've approved this before but is this is going to be pretty nasty for programs that create 1000 topics this mutex will create a global chokepoint if there wasn't one already, so they will be literally created one at a time?

Do we just need locks for a particular topic? in that case maybe we could have a sync.Map of murexes indexed by topic ARN here so as not to conflict globally?

CC @VenelinMartinov

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@t0yv0 I've updated this so that the lock is based on the topic arn. I think we only care to prevent race conditions if the topics have the same name.

t0yv0
t0yv0 approved these changes Apr 10, 2024
View reviewed changes
Copy link
Member

@t0yv0 t0yv0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good but recommend landing the upstream upgrade first as patch merging is a mess.

@corymhall corymhall closed this Apr 11, 2024
@VenelinMartinov
Copy link
Contributor

Hey folks, I might have missed a discussion here - why'd you opt to close this?

@corymhall
Copy link
Contributor Author

Hey folks, I might have missed a discussion here - why'd you opt to close this?

I messed up trying to rebase the patches from master so I just recreated it #3809

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@t0yv0 t0yv0 t0yv0 approved these changes

Assignees

@corymhall corymhall

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Implicit importing of SNS Topics
3 participants
@corymhall @VenelinMartinov @t0yv0