Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Image garbage collection is always disabled in kubelet in 1.11 #66046

Closed
yujuhong opened this issue Jul 10, 2018 · 4 comments · Fixed by #66051
Closed

Image garbage collection is always disabled in kubelet in 1.11 #66046

yujuhong opened this issue Jul 10, 2018 · 4 comments · Fixed by #66051
Labels
kind/bug Categorizes issue or PR as related to a bug. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. sig/node Categorizes an issue or PR as relevant to SIG Node.
Milestone
v1.11

Comments

@yujuhong
Copy link
Contributor

In the StartGarbageCollection function, there is a defer close(stopChan) that always runs and closes the channel when returning. This means that instead of running periodically, the image garbage collection will be terminated immediately when the function returns.

kubernetes/pkg/kubelet/kubelet.go

Lines 1247 to 1253 in 91e7b4f

stopChan := make(chan struct{})
defer close(stopChan)
// when the high threshold is set to 100, stub the image GC manager
if kl.kubeletConfiguration.ImageGCHighThresholdPercent == 100 {
glog.V(2).Infof("ImageGCHighThresholdPercent is set 100, Disable image GC")
go func() { stopChan <- struct{}{} }()
}

The bug was introduced by #51423 in 1.11

/cc @derekwaynecarr @jiaxuanzhou @kubernetes/sig-node-bugs @dashpole
@yujuhong yujuhong added kind/bug Categorizes issue or PR as related to a bug. sig/node Categorizes an issue or PR as relevant to SIG Node. labels Jul 10, 2018
@yujuhong yujuhong added this to the v1.11 milestone Jul 10, 2018
@yujuhong yujuhong added the priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. label Jul 10, 2018
@k8s-github-robot
Copy link

[MILESTONENOTIFIER] Milestone Issue: Up-to-date for process

@yujuhong

Issue Labels
  • sig/node: Issue will be escalated to these SIGs if needed.
  • priority/critical-urgent: Never automatically move issue out of a release milestone; continually escalate to contributor and SIG through all available channels.
  • kind/bug: Fixes a bug discovered during the current release.
Help

@yujuhong
Copy link
Contributor Author

yujuhong commented Jul 10, 2018 •
edited
Loading

BTW, when the threshold is set to 100% (the use case that the PR originally wanted to fix), the code'd try sending to a closed channel and triggers a panic.

@jiaxuanzhou
Copy link
Contributor

jiaxuanzhou commented Jul 11, 2018 •
edited
Loading

@yujuhong i will check it soon, will submit one patch immediately, thanks

@jiaxuanzhou jiaxuanzhou mentioned this issue Jul 11, 2018
Merged
k8s-github-robot pushed a commit that referenced this issue Jul 11, 2018
Merge pull request #66051 from jiaxuanzhou/imageGC
6c6449c 
Automatic merge from submit-queue (batch tested with PRs 66051, 66049). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix:Image garbage collection is always disabled in kubelet in 1.11 

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes  #66046


**Special notes for your reviewer**:
@yujuhong @derekwaynecarr 
**Release note**:

```release-note
NONE
```
@thanasisk
Copy link

does this bug applies to v1.11.2 ? or v.1.11.2 includes the fix?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Categorizes issue or PR as related to a bug. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. sig/node Categorizes an issue or PR as relevant to SIG Node.
Projects
None yet
Milestone
v1.11
Development

Successfully merging a pull request may close this issue.

Fix:Image garbage collection is always disabled in kubelet in 1.11 jiaxuanzhou/kubernetes
4 participants
@thanasisk @jiaxuanzhou @yujuhong @k8s-github-robot