Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DXF: support modify task concurrency and business params at runtime #57497

Open
D3Hunter opened this issue Nov 19, 2024 · 0 comments
Open

DXF: support modify task concurrency and business params at runtime #57497

D3Hunter opened this issue Nov 19, 2024 · 0 comments
Labels
type/feature-request Categorizes issue or PR as related to a new feature.

Comments

@D3Hunter
Copy link
Contributor

D3Hunter commented Nov 19, 2024

Feature Request

Is your feature request related to a problem? Please describe:

Currently, after a task is submitted to DXF, we cannot change the max resource it can be used at runtime. But we need this feature in some cases, such as when submitting alter table xxx add index, current value of tidb_ddl_reorg_worker_cnt of which default is 4, is used as the task concurrency, later users might want to accelerate it by increasing the concurrency, which is a supported feature when adding index in standalone mode. And import-into might need it too.

Also we need to modify some business specific param at runtime, such as batch-size and max-write-speed for add-index.

Describe the feature you'd like:

support modify task concurrency at runtime, we want to make it easier to extend as we might need to modify other task config at runtime later. and support modify business specific param

Describe alternatives you've considered:

this is new feature
Teachability, Documentation, Adoption, Migration Strategy:

DXF is the underlying framework behind upper business, including add-index and import-into, so to support change concurrency or business specific param for upper business, DXF need to support it first, then upper layer can integrate the ability and expose some API to user

@D3Hunter D3Hunter added the type/feature-request Categorizes issue or PR as related to a new feature. label Nov 19, 2024
ti-chi-bot bot pushed a commit that referenced this issue Nov 21, 2024
ti-chi-bot bot pushed a commit that referenced this issue Nov 28, 2024
@D3Hunter D3Hunter changed the title DXF: support modify task concurrency at runtime DXF: support modify task concurrency and business param at runtime Dec 2, 2024
@D3Hunter D3Hunter changed the title DXF: support modify task concurrency and business param at runtime DXF: support modify task concurrency and business params at runtime Dec 2, 2024
ti-chi-bot bot pushed a commit that referenced this issue Dec 10, 2024
ti-chi-bot bot pushed a commit that referenced this issue Dec 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature-request Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

No branches or pull requests

1 participant