-
Notifications
You must be signed in to change notification settings - Fork 5.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DXF: support modify task concurrency and business params at runtime #57497
Labels
type/feature-request
Categorizes issue or PR as related to a new feature.
Comments
D3Hunter
added
the
type/feature-request
Categorizes issue or PR as related to a new feature.
label
Nov 19, 2024
13 tasks
This was referenced Nov 20, 2024
This was referenced Nov 25, 2024
13 tasks
D3Hunter
changed the title
DXF: support modify task concurrency at runtime
DXF: support modify task concurrency and business param at runtime
Dec 2, 2024
D3Hunter
changed the title
DXF: support modify task concurrency and business param at runtime
DXF: support modify task concurrency and business params at runtime
Dec 2, 2024
13 tasks
13 tasks
This was referenced Dec 11, 2024
Merged
13 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Feature Request
Is your feature request related to a problem? Please describe:
Currently, after a task is submitted to DXF, we cannot change the max resource it can be used at runtime. But we need this feature in some cases, such as when submitting alter table xxx add index, current value of tidb_ddl_reorg_worker_cnt of which default is 4, is used as the task concurrency, later users might want to accelerate it by increasing the concurrency, which is a supported feature when adding index in standalone mode. And import-into might need it too.
Also we need to modify some business specific param at runtime, such as
batch-size
andmax-write-speed
for add-index.Describe the feature you'd like:
support modify task concurrency at runtime, we want to make it easier to extend as we might need to modify other task config at runtime later. and support modify business specific param
Describe alternatives you've considered:
this is new feature
Teachability, Documentation, Adoption, Migration Strategy:
DXF is the underlying framework behind upper business, including add-index and import-into, so to support change concurrency or business specific param for upper business, DXF need to support it first, then upper layer can integrate the ability and expose some API to user
The text was updated successfully, but these errors were encountered: