Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug Report: Vttbalet memory spikes after upgarde to v16 during backup to AZ blob storage #13771

Closed
gairadzi opened this issue Aug 12, 2023 · 1 comment · Fixed by #13770
Closed

Bug Report: Vttbalet memory spikes after upgarde to v16 during backup to AZ blob storage #13771

gairadzi opened this issue Aug 12, 2023 · 1 comment · Fixed by #13770
Assignees
@mattlord
Labels
Component: Backup and Restore Type: Bug

Comments

@gairadzi
Copy link

Overview of the Issue

After upgrading our vitess cluster to v16.0.2 we see that when issuing the Backup command on a tablet the vttablet container's memory spikes (the size of the DB being backup up has little effect on the total memory needed).

Reproduction Steps

Run vtctlclient Backup <TABLET_NAME> and check the memory usage on the vttablet container in that pod:
image

Including the allocs & heap debug profiles from the vttablet during the backup:
allocs.zip
heap.zip

Binary Version

Using the `vitess/lite:v16.0.2-mysql80` vitess images

Operating System and Environment details

Using the `vitess/lite:v16.0.2-mysql80` vitess images

Log Fragments

No response
@gairadzi gairadzi added Needs Triage This issue needs to be correctly labelled and triaged Type: Bug labels Aug 12, 2023
@mattlord mattlord mentioned this issue Aug 12, 2023
Merged
4 tasks
@mattlord mattlord removed the Needs Triage This issue needs to be correctly labelled and triaged label Aug 12, 2023
@mattlord mattlord self-assigned this Aug 12, 2023
@shlomi-noach
Copy link
Contributor

v17 backport: #13775
v16 backport: #13774

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mattlord mattlord

Labels
Component: Backup and Restore Type: Bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Address vttablet memory usage with backups to Azure Blob Service planetscale/vitess
3 participants
@mattlord @shlomi-noach @gairadzi