Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

vendor: update cAdvisor to v0.38.6 #97174

Merged
merged 1 commit into from Dec 11, 2020
Merged

vendor: update cAdvisor to v0.38.6 #97174

merged 1 commit into from Dec 11, 2020

Conversation

hanlins
Copy link
Member

@hanlins hanlins commented Dec 9, 2020 •
edited

What type of PR is this?
/kind bug

What this PR does / why we need it:
Bug fix for cAdvisor exponential backoff. Originally backoff multiplier defaults to zero, causing backoff timeout drop to zero after the first retry, causing retries flooding. Before the fix, kubelet experienced issue that CPU will spike to 100% after restarting containerd.

Which issue(s) this PR fixes:

Fixes #95727

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Fixed a bug in kubelet that will saturate CPU utilization after containerd got restarted.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

cc @bobbypage

@hanlins
vendor: update cAdvisor to v0.38.6
26d02be 
Signed-off-by: Hanlin Shi <shihanlin9@gmail.com>
@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/bug Categorizes issue or PR as related to a bug. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Dec 9, 2020
@k8s-ci-robot
Copy link
Contributor

@hanlins: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label Dec 9, 2020
@k8s-ci-robot
Copy link
Contributor

Hi @hanlins. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Dec 9, 2020
@k8s-ci-robot k8s-ci-robot added the area/dependency Issues or PRs related to dependency changes label Dec 9, 2020
@bobbypage
Copy link
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Dec 10, 2020
@bobbypage
Copy link
Member

/sig node

@k8s-ci-robot k8s-ci-robot added sig/node Categorizes an issue or PR as relevant to SIG Node. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Dec 10, 2020
@bobbypage
Copy link
Member

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 10, 2020
@hanlins
Copy link
Member Author

hanlins commented Dec 10, 2020

/retest

@bobbypage
Copy link
Member

/assign @dims @liggitt

This fixes #95727 where kubelet CPU usage will increase to > 100% cpu and kubelet will flood the logs when containerd service is stopped. We should get this in, especially considering more users will be migrating to containerd as part of the recent dockershim depreciation.

We also want to cherrypick this into 1.20 & 1.19. @hanlins created (#97175) and 1.19 (#97176) respectively.

@bobbypage bobbypage mentioned this pull request Dec 10, 2020
Closed
@bobbypage
Copy link
Member

/retest

@dims
Copy link
Member

dims commented Dec 10, 2020

@hanlins thanks for the debugging/fix :)
@liggitt i'd support backporting this to 1.20.x

/approve
/lgtm

@dims
Copy link
Member

dims commented Dec 10, 2020

/assign @smarterclayton

@bobbypage
Copy link
Member

/cc @dchen1107

@dchen1107
Copy link
Member

/lgtm
/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dchen1107, dims, hanlins

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 11, 2020
@k8s-ci-robot k8s-ci-robot merged commit 73118ae into kubernetes:master Dec 11, 2020
@k8s-ci-robot k8s-ci-robot added this to the v1.21 milestone Dec 11, 2020
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Dec 12, 2020
@github-actions github-actions bot mentioned this pull request Dec 16, 2020
Open
@hanlins hanlins mentioned this pull request Feb 16, 2021
Closed
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@brendandburns brendandburns Awaiting requested review from brendandburns

@dchen1107 dchen1107 Awaiting requested review from dchen1107

Assignees

@dims dims

@bobbypage bobbypage

@liggitt liggitt

@smarterclayton smarterclayton

@dchen1107 dchen1107

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/dependency Issues or PRs related to dependency changes cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/node Categorizes an issue or PR as relevant to SIG Node. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Milestone
v1.21
Development

Successfully merging this pull request may close these issues.

The cpu of kubelet is always 100% after containerd.service restart
7 participants
@hanlins @k8s-ci-robot @bobbypage @dims @dchen1107 @liggitt @smarterclayton