Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

skip store limit for empty regions created by lightning #6951

Open
mittalrishabh opened this issue Aug 11, 2023 · 4 comments
Open

skip store limit for empty regions created by lightning #6951

mittalrishabh opened this issue Aug 11, 2023 · 4 comments
Labels
type/enhancement The issue or PR belongs to an enhancement.

Comments

@mittalrishabh
Copy link
Contributor

Enhancement Task

Lightning creates thousands of empty regions. Lightning hit the store limit very often and it sometimes lead to skewed regions.
We should skip the store limit if scatter on empty regions are inexpensive.

@mittalrishabh mittalrishabh added the type/enhancement The issue or PR belongs to an enhancement. label Aug 11, 2023
@rleungx
Copy link
Member

rleungx commented Aug 11, 2023

We have added an option to skip the limit check: #6593

@mittalrishabh
Copy link
Contributor Author

I want to understand how it impacts tikv if there are lot of pending scatter operations. Do you have a write up on it

@rleungx
Copy link
Member

rleungx commented Aug 14, 2023

If we don't scatter the regions or the scatter operation is pending/canceled, the import job may have hotspots on TiKV. Most of the import flow might happen on the three TiKVs if we only use the 3 replicas setting, which affects its progress and the data might be skewed on TiKVs.

@mittalrishabh
Copy link
Contributor Author

Lightning with parallel import can cause split operations on non empty regions. I want to understand the implication of that.

@bufferflies bufferflies changed the title skip store limit for emptry regions created by lightning skip store limit for empty regions created by lightning Dec 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

No branches or pull requests

2 participants