Skip to content

Commit

Permalink
feat: added sharding_config field in DocumentOutputConfig.GcsOutputCo…
Browse files Browse the repository at this point in the history
…nfig in document_io.proto (#3735)

- [ ] Regenerate this pull request now.

PiperOrigin-RevId: 495051185

Source-Link: https://togithub.com/googleapis/googleapis/commit/0ff0b240ca1026d914e2256d4a32f885d1a31c54

Source-Link: https://togithub.com/googleapis/googleapis-gen/commit/ffa8b5f51bd4258481b8fc73ca7370f78847d5d7
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWNvbnRlbnR3YXJlaG91c2UvLk93bEJvdC55YW1sIiwiaCI6ImZmYThiNWY1MWJkNDI1ODQ4MWI4ZmM3M2NhNzM3MGY3ODg0N2Q1ZDcifQ==
  • Loading branch information
gcf-owl-bot[bot] authored Dec 13, 2022
1 parent c7a240f commit 1b3b2b2
Showing 1 changed file with 12 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -74,13 +74,25 @@ message BatchDocumentsInputConfig {
message DocumentOutputConfig {
// The configuration used when outputting documents.
message GcsOutputConfig {
// The sharding config for the output document.
message ShardingConfig {
// The number of pages per shard.
int32 pages_per_shard = 1;

// The number of overlapping pages between consecutive shards.
int32 pages_overlap = 2;
}

// The Cloud Storage uri (a directory) of the output.
string gcs_uri = 1;

// Specifies which fields to include in the output documents.
// Only supports top level document and pages field so it must be in the
// form of `{document_field_name}` or `pages.{page_field_name}`.
google.protobuf.FieldMask field_mask = 2;

// Specifies the sharding config for the output document.
ShardingConfig sharding_config = 3;
}

// The destination of the results.
Expand Down

0 comments on commit 1b3b2b2

Please sign in to comment.