Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backupccl: allow SSTBatcher to split ranges during RESTORE #83144

Closed
msbutler opened this issue Jun 21, 2022 · 2 comments
Closed

backupccl: allow SSTBatcher to split ranges during RESTORE #83144

msbutler opened this issue Jun 21, 2022 · 2 comments
Assignees
Labels
A-disaster-recovery C-performance Perf of queries or internals. Solution not expected to change functional behavior. T-disaster-recovery

Comments

@msbutler
Copy link
Collaborator

msbutler commented Jun 21, 2022

Currently, during RESTORE, the SSTBatcher is not configured to split a range it's ingesting. This isn't a big deal because the ranges RESTORE creates and writes data to tend to be under-filled. Specifically, a single SSTBatcher in RESTORE ingests a single restoreSpanEntry's data into a single range, but the amount of data associated with a restoreSpanEntry tends to be much smaller than the default range size. However, once #83139 implements restoreSpanEntry target sizes, an SSTBatcher may fill multiple ranges worth of data. To prevent post restore splits, the SSTBatcher should split the range its ingesting once it has ingested a range's worth of data.

Since the SSTBatcher operates on a specific node, these SSTBatcher splits would also prevent any post-Restore splits on a cluster that has zone configs that specify different target range sizes for different zones.

Jira issue: CRDB-16887

Epic CRDB-10127

@msbutler msbutler added C-performance Perf of queries or internals. Solution not expected to change functional behavior. T-disaster-recovery labels Jun 21, 2022
@msbutler msbutler self-assigned this Jun 21, 2022
@blathers-crl
Copy link

blathers-crl bot commented Jun 21, 2022

cc @cockroachdb/bulk-io

@msbutler
Copy link
Collaborator Author

Fixed by #86496

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-disaster-recovery C-performance Perf of queries or internals. Solution not expected to change functional behavior. T-disaster-recovery
Projects
No open projects
Archived in project
Development

No branches or pull requests

1 participant