Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EventHubs] Finalize design on Idempotent Producer #16994

Closed
yunhaoling opened this issue Mar 1, 2021 · 2 comments
Closed

[EventHubs] Finalize design on Idempotent Producer #16994

yunhaoling opened this issue Mar 1, 2021 · 2 comments
Assignees
Labels
Client This issue points to a problem in the data-plane of the library. Event Hubs Messaging Messaging crew
Milestone

Comments

@yunhaoling
Copy link
Contributor

yunhaoling commented Mar 1, 2021

background

idempotent producer is targeted to solve the problem of sending duplicate messages and provide "exactly once delivery".
There has been a preview release for idempotent producer already which is available on the branch: https://github.com/Azure/azure-sdk-for-python/tree/feature/eventhub/idempotent-producer

  • latest update: discussion postponed, currently no feature request from python users, more data is needed
  • This issue also has something to do with the StreamingProducer Design: [EventHubs] Streaming Producer Design #18718.
    We need to take both into account when making design decision.
  • The buffered producer design is settled, it's a question of how the idempotency works for buffered mode.
  • There is an ongoing email thread for discussion within the team, so need to have a cross-language agreement on the two concepts/whether to support it/overall structure.

Another design topic would be: Idempotent Producer naming of *_published_sequence_number property on different Objects

we have two prefixs on *_published_sequence_number on EventDataBatch and EventHubProducerClient, and we have multiple opinions on the naming:

class EventDataBatch:
    def starting_published_sequence_number(self):
    # beginning/first/ or just published_sequence_number?
# `PartitionPublishingProperties` which is a dictionary returned by `EventHubProducerClient.get_partition_publishing_properties()`

dic['last_published_sequence_number']
# ending_published_sequence_number?

tasks

  • sync up with Jesse/service team first, definite the scope
@ghost ghost added the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Mar 1, 2021
@yunhaoling yunhaoling self-assigned this Mar 1, 2021
@yunhaoling yunhaoling added Client This issue points to a problem in the data-plane of the library. Event Hubs labels Mar 1, 2021
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Mar 1, 2021
@yunhaoling yunhaoling added this to the [2021] April milestone Mar 1, 2021
@yunhaoling yunhaoling modified the milestones: [2021] April, [2021] May Mar 31, 2021
@lmazuel lmazuel added the Messaging Messaging crew label Apr 12, 2021
@yunhaoling yunhaoling changed the title [EventHubs] Discussion: Idempotent Producer naming of *_published_sequence_number property on different Objects [EventHubs] Finalize design on Idempotent Producer May 13, 2021
@lmazuel lmazuel modified the milestones: [2021] May, Backlog May 19, 2021
@yunhaoling yunhaoling modified the milestones: Backlog, [2021] July Jun 14, 2021
@yunhaoling yunhaoling assigned kashifkhan and unassigned yunhaoling Apr 13, 2022
@jsquire
Copy link
Member

jsquire commented Jul 14, 2022

@kashifkhan - If we can confirm there's a single option for "EnableIdempotentPartitions" that the buffered producer can use - then the design is final and we can kill this one. 😄

Copy link

Hi @yunhaoling, we deeply appreciate your input into this project. Regrettably, this issue has remained inactive for over 2 years, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Mar 15, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Mar 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Client This issue points to a problem in the data-plane of the library. Event Hubs Messaging Messaging crew
Projects
None yet
Development

No branches or pull requests

4 participants