Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce the memory usage of huge transaction #18308

Open
zhangjinpeng87 opened this issue Jul 1, 2020 · 1 comment
Open

Reduce the memory usage of huge transaction #18308

zhangjinpeng87 opened this issue Jul 1, 2020 · 1 comment
Assignees
@zhangjinpeng87
Labels
feature/accepted This feature request is accepted by product managers priority/P0 The issue has P0 priority. type/feature-request Categorizes issue or PR as related to a new feature.
Milestone
v5.0.0 ga

Comments

@zhangjinpeng87
Copy link
Contributor

zhangjinpeng87 commented Jul 1, 2020 •
edited
Loading

Feature Request

Is your feature request related to a problem? Please describe:

A 10GB transaction may eat ~60GB memory in tidb-server, it means huge transaction can't use in a machine with 32GB or 16GB. We need to reduce the memory usage to a reasonable size.

Describe the feature you'd like:

  • A tidb-server with 16GB memory can use huge transaction and will not trigger OOM issue

Describe alternatives you've considered:

  • Open swap to relieve the memory pressure, but it will slowdown whole system, also in k8s environment, there is no way to open swap.

Teachability, Documentation, Adoption, Migration Strategy:

This is an sub-task of #17479

Time

GanttStart: 2020-07-15
GanttDue: 2020-08-15
GanttProgress: 50%
@zhangjinpeng87 zhangjinpeng87 added the type/feature-request Categorizes issue or PR as related to a new feature. label Jul 1, 2020
@jackysp jackysp mentioned this issue Jul 3, 2020
Open
4 tasks
@zhangjinpeng87 zhangjinpeng87 mentioned this issue Jul 9, 2020
Closed
@scsldb scsldb added the feature/reviewing This feature request is reviewing by product managers label Jul 16, 2020
@zhangjinpeng87
Copy link
Contributor Author

zhangjinpeng87 commented Jul 16, 2020 •
edited
Loading

We can optimize the memory usage for 10GB transactions in the first step. @bobotu

@zhangjinpeng87 zhangjinpeng87 added feature/accepted This feature request is accepted by product managers and removed feature/reviewing This feature request is reviewing by product managers labels Jul 16, 2020
@zhangjinpeng87 zhangjinpeng87 added this to the v5.0.0-alpha milestone Jul 16, 2020
@scsldb scsldb added the priority/P0 The issue has P0 priority. label Jul 17, 2020
@zhangjinpeng87 zhangjinpeng87 changed the title Enlarge huge transaction size limitation and prevent OOM Reduce the memory usage of huge transaction Jul 30, 2020
@bobotu bobotu removed their assignment Dec 31, 2020
@jebter jebter modified the milestones: v5.0.0-alpha, v5.0.0-rc Jan 7, 2021
@jebter jebter modified the milestones: v5.0.0-rc, v5.0.0 Jan 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zhangjinpeng87 zhangjinpeng87

Labels
feature/accepted This feature request is accepted by product managers priority/P0 The issue has P0 priority. type/feature-request Categorizes issue or PR as related to a new feature.
Projects
None yet
Milestone
v5.0.0 ga
Development

No branches or pull requests
4 participants
@jebter @scsldb @bobotu @zhangjinpeng87