Skip to content

Commit

Permalink
[YAML] Note about error output schemas.
Browse files Browse the repository at this point in the history
  • Loading branch information
robertwb committed Nov 2, 2023
1 parent dc24213 commit 07c8bc4
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions sdks/python/apache_beam/yaml/yaml_errors.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,10 @@ Note that with `error_handling` declared, `MapToFields.my_error_output`
logging the bad records to stdout would be sufficient (though not recommended
for a robust pipeline).

Note also that the exact format of the error outputs is still being finalized.
They can be safely printed and written to outputs, but their precise schema
may change in a future version of Beam and should not yet be depended on.

Some transforms allow for extra arguments in their error_handling config, e.g.
for Python functions one can give a `threshold` which limits the relative number
of records that can be bad before considering the entire pipeline a failure
Expand Down

0 comments on commit 07c8bc4

Please sign in to comment.