-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dataflow runner specialization of PubsubIO should validate messages #21252
Comments
Unable to assign user @scwhittle. If able, self-assign, otherwise tag @damccorm so that he can assign you. Because of GitHub's spam prevention system, your activity is required to enable assignment in this repo. |
@damccorm Hi Danny, can you assign this to me? I don't seem to be able to though I have committed to the repo before. |
One of the unfortunate downsides of moving to issues is that non-committers can't self-assign/triage issues that they didn't create by default. I think that's a workflow we'll need to support going forward though, I'll look into it. For the moment, I'd recommend commenting/declaring intent to work on any issues. Once #21719 is in, you should be able to self assign and close issues with chat-op commands |
… SDK This improves visiblity as otherwise messages can become stuck publishing inside the dataflow runner. This also allows users to handle errors by catching them. Fixes issue apache#21252 for python sdk.
…hing limits in the python SDK * [#21252] Enforce pubsub message publishing limits in the python SDK This improves visiblity as otherwise messages can become stuck publishing inside the dataflow runner. This also allows users to handle errors by catching them. Fixes issue #21252 for python sdk. * fixtest Co-authored-by: Pablo <[email protected]>
.take-issue |
.close-issue |
…ssage publishing limits in the python SDK * [apache#21252] Enforce pubsub message publishing limits in the python SDK This improves visiblity as otherwise messages can become stuck publishing inside the dataflow runner. This also allows users to handle errors by catching them. Fixes issue apache#21252 for python sdk. * fixtest Co-authored-by: Pablo <[email protected]>
Currently invalid messages, such as those with attributes exceeding the maximum size are processed by the bundle successfully but fail to commit.
Throwing an exception when trying to write such a message directly would increase visibility as well as allowing users to catch and handle such exceptions.
Imported from Jira BEAM-12942. Original Jira may contain additional context.
Reported by: scwhittle.
The text was updated successfully, but these errors were encountered: