Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make all health checks probing consistent #98376

Merged
merged 4 commits into from Mar 6, 2021
Merged

Make all health checks probing consistent #98376

merged 4 commits into from Mar 6, 2021

Conversation

matthyx
Copy link
Contributor

@matthyx matthyx commented Jan 25, 2021 •
edited

What type of PR is this?
/kind bug
/kind cleanup

What this PR does / why we need it:
Current handling of readiness/startupProbe result updates don't include an immediate sync after declaring the container as failed.
Thus, the sync is done via slower async methods.

Which issue(s) this PR fixes:

Fixes #96731
Fixes #98007

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Faster update the pod status reacting on probe results. Among other changes, the first readiness probe will be called faster when startup probes succeeded, which will make Pod status as ready faster.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jan 25, 2021
@k8s-ci-robot k8s-ci-robot added area/kubelet area/test sig/node Categorizes an issue or PR as relevant to SIG Node. sig/testing Categorizes an issue or PR as relevant to SIG Testing. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jan 25, 2021
@matthyx
Copy link
Contributor Author

matthyx commented Jan 25, 2021

/cc @SergeyKanzhelev

@matthyx matthyx mentioned this pull request Jan 25, 2021
Closed
@matthyx
Copy link
Contributor Author

matthyx commented Jan 25, 2021

/test pull-kubernetes-e2e-gce-ubuntu-containerd

@matthyx matthyx changed the title WIP make all health checks probing consistent Make all health checks probing consistent Jan 25, 2021
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 25, 2021
@matthyx matthyx mentioned this pull request Jan 25, 2021
Closed
@matthyx
Copy link
Contributor Author

matthyx commented Jan 25, 2021

/assign @timothysc

@SergeyKanzhelev
Copy link
Member

labels cleanup:

/kind bug
/kind cleanup
/priority backlog
/triage accepted

Are you sure you don't want to add a release note? May be handy

@k8s-ci-robot k8s-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Jan 25, 2021
BenTheElder
BenTheElder approved these changes Mar 6, 2021
View reviewed changes
Copy link
Member

@BenTheElder BenTheElder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve
I took getting some sleep seriously 🙃

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 6, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BenTheElder, matthyx, mrunalp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit c193c1b into kubernetes:master Mar 6, 2021
SIG Node PR Triage automation moved this from Needs Approver to Done Mar 6, 2021
@howardjohn
Copy link
Contributor

This may deserve a release note given it will impact (in a positive way) how probes work

@matthyx
Copy link
Contributor Author

matthyx commented Mar 7, 2021

This may deserve a release note given it will impact (in a positive way) how probes work

Is it possible to add one in the description after merge, or it won't work and I need to do it differently?

@matthyx matthyx deleted the mega branch March 7, 2021 19:18
@howardjohn
Copy link
Contributor

howardjohn commented Mar 7, 2021 via email

@annajung
Copy link

annajung commented Mar 7, 2021

You might need to make a PR to k/sig-release release note draft https://github.com/kubernetes/sig-release/tree/master/releases/release-1.21/release-notes, but I would double-check with @wilsonehusin, 1.21 Release notes lead.

@wilsonehusin, Could you help answer if the description change after merge would work?

@wilsonehusin
Copy link
Contributor

you can modify this PR body anytime before we start collecting. we will be gathering the next batch on Tuesday / whenever beta.1 releases 😄

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Mar 8, 2021
@matthyx
Copy link
Contributor Author

matthyx commented Mar 8, 2021

you can modify this PR body anytime before we start collecting. we will be gathering the next batch on Tuesday / whenever beta.1 releases

Thanks @wilsonehusin I have added something, please let me know if that's clear enough.

@SergeyKanzhelev
Copy link
Member

Maybe more text like this:

Faster update the pod status reacting on probe results. Among other changes, the first readiness probe will be called faster when startup probes succeeded, which will make Pod status as ready faster.

@wilsonehusin
Copy link
Contributor

I usually use the heuristic of "giving reader a good-enough idea of what happened without opening the PR". From glancing through the issues mentioned, do you think the following would be accurate to describe the changes?

First readiness probe updates Pod status faster, therefore Pods may reach ready state faster than the time defined in readinessProbe.

@ehashman ehashman mentioned this pull request Mar 9, 2021
Merged
4 tasks
@matthyx matthyx mentioned this pull request Mar 9, 2021
Merged
This was referenced Mar 24, 2021
Closed
Closed
Closed
@ykulazhenkov ykulazhenkov mentioned this pull request Apr 13, 2021
Closed
@matthyx matthyx mentioned this pull request Jun 9, 2021
Closed
@wackxu wackxu mentioned this pull request Mar 10, 2022
Merged
@adrianludwin adrianludwin mentioned this pull request Mar 29, 2022
Closed
@liushi001 liushi001 mentioned this pull request Jul 7, 2022
Open
This was referenced Jul 4, 2023
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SergeyKanzhelev SergeyKanzhelev SergeyKanzhelev left review comments

@ehashman ehashman ehashman requested changes

@BenTheElder BenTheElder BenTheElder approved these changes

@feiskyer feiskyer Awaiting requested review from feiskyer

@krmayankk krmayankk Awaiting requested review from krmayankk

Assignees

@timothysc timothysc

@BenTheElder BenTheElder

@ehashman ehashman

@derekwaynecarr derekwaynecarr

@SergeyKanzhelev SergeyKanzhelev

@yujuhong yujuhong

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/e2e-test-framework Issues or PRs related to refactoring the kubernetes e2e test framework area/kubelet area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
SIG Node CI/Test Board
Triage
SIG Node CI/Test Board
Archived in project
SIG Node PR Triage
Done
Milestone
v1.21
Development

Successfully merging this pull request may close these issues.

startupProbe takes 60s before killing a failing container Dynamic readiness probe period time