Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding new EndpointsOverCapacity annotation for Endpoints controller #99975

Merged
merged 1 commit into from Mar 10, 2021
Merged

Adding new EndpointsOverCapacity annotation for Endpoints controller #99975

merged 1 commit into from Mar 10, 2021

Conversation

robscott
Copy link
Member

@robscott robscott commented Mar 9, 2021

What type of PR is this?

/kind cleanup
/kind feature

What this PR does / why we need it:

Now that the EndpointSlice API and controllers are GA, the Endpoints controller will use this annotation to warn when Endpoints are over capacity. In a future release, this warning will be replaced with truncation.

Special notes for your reviewer:

This represents the last item in the v1.21 roll out plan for the EndpointSlice KEP.

Does this PR introduce a user-facing change?

The Endpoints controller will now set the `endpoints.kubernetes.io/over-capacity` annotation to "warning" when an Endpoints resource contains more than 1000 addresses. In a future release, the controller will truncate Endpoints that exceed this limit. The EndpointSlice API can be used to support significantly larger number of addresses.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

/sig network
/priority important-soon
/triage accepted
/cc @aojea @swetharepakula
/assign @thockin

@k8s-ci-robot k8s-ci-robot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Mar 9, 2021
@k8s-ci-robot k8s-ci-robot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. kind/feature Categorizes issue or PR as related to a new feature. labels Mar 9, 2021
@k8s-ci-robot
Copy link
Contributor

@robscott: GitHub didn't allow me to request PR reviews from the following users: swetharepakula.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

What type of PR is this?

/kind cleanup
/kind feature

What this PR does / why we need it:

Now that the EndpointSlice API and controllers are GA, the Endpoints controller will use this annotation to warn when Endpoints are over capacity. In a future release, this warning will be replaced with truncation.

Special notes for your reviewer:

This represents the last item in the v1.21 roll out plan for the EndpointSlice KEP.

Does this PR introduce a user-facing change?

The Endpoints controller will now set the `endpoints.kubernetes.io/over-capacity` annotation to "warning" when an Endpoints resource contains more than 1000 addresses. In a future release, the controller will truncate Endpoints that exceed this limit. The EndpointSlice API can be used to support significantly larger number of addresses.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

/sig network
/priority important-soon
/triage accepted
/cc @aojea @swetharepakula
/assign @thockin

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the sig/network Categorizes an issue or PR as relevant to SIG Network. label Mar 9, 2021
@k8s-ci-robot k8s-ci-robot requested a review from aojea March 9, 2021 02:03
@k8s-ci-robot k8s-ci-robot added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. triage/accepted Indicates an issue or PR is ready to be actively worked on. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API sig/apps Categorizes an issue or PR as relevant to SIG Apps. and removed do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Mar 9, 2021
@fejta-bot
Copy link

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

swetharepakula
swetharepakula approved these changes Mar 9, 2021
View reviewed changes
Copy link
Member

@swetharepakula swetharepakula left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

thockin
thockin reviewed Mar 9, 2021
View reviewed changes
Copy link
Member

@thockin thockin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

pkg/apis/core/annotation_key_constants.go Outdated
// EndpointsOverCapacity will be set on an Endpoints resource when it
// exceeds the maximum capacity of 1000 addresses. Inititially the Endpoints
// controller will set this annotation with a value of "warning". In a
// future release, the controller will set this annotation with a value of
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/will/may ? I am not 100% sure that we WILL - I think it depends on data

Copy link
Member Author

@robscott robscott Mar 9, 2021 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, "may" is better than "will" here, updated. I personally think we should enforce some kind of limit here, maybe 1k is too low, but if we don't enforce some kind of limit, the Endpoints controller is just going to keep retrying endlessly when it hits the etcd object size limit. Without some kind of limit here, we're limiting how far EndpointSlices can go with this controller enabled.

pkg/controller/endpoint/endpoints_controller.go Outdated Show resolved Hide resolved
staging/src/k8s.io/api/core/v1/annotation_key_constants.go Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 9, 2021
@robscott
Adding new EndpointsOverCapacity annotation for Endpoints controller
8a3f720 
Now that the EndpointSlice API and controllers are GA, the Endpoints
controller will use this annotation to warn when Endpoints are over
capacity. In a future release, this warning will be replaced with
truncation.
@robscott
Copy link
Member Author

robscott commented Mar 9, 2021

@thockin Thanks for the review! Replaced "will" with "may", PTAL.

thockin
thockin reviewed Mar 9, 2021
View reviewed changes
Copy link
Member

@thockin thockin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 9, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: robscott, swetharepakula, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 05c4feb into kubernetes:master Mar 10, 2021
@k8s-ci-robot k8s-ci-robot added this to the v1.21 milestone Mar 10, 2021
@annajung annajung mentioned this pull request Mar 10, 2021
Closed
@swetharepakula swetharepakula mentioned this pull request Mar 15, 2021
Merged
valyala added a commit to VictoriaMetrics/VictoriaMetrics that referenced this pull request Oct 19, 2021
@valyala
lib/promscrape/discovery/kubernetes: log a warning if `role: endpoint…
3408a05 
…s` discovers more than 1000 targets per a single endpoint

In this case `role: endpointslice` must be used instead.

See the following references:

* https://kubernetes.io/docs/reference/labels-annotations-taints/#endpoints-kubernetes-io-over-capacity
* kubernetes/kubernetes#99975
* prometheus/prometheus#7572 (comment)
valyala added a commit to VictoriaMetrics/VictoriaMetrics that referenced this pull request Oct 19, 2021
@valyala
lib/promscrape/discovery/kubernetes: log a warning if `role: endpoint…
ea69eef 
…s` discovers more than 1000 targets per a single endpoint

In this case `role: endpointslice` must be used instead.

See the following references:

* https://kubernetes.io/docs/reference/labels-annotations-taints/#endpoints-kubernetes-io-over-capacity
* kubernetes/kubernetes#99975
* prometheus/prometheus#7572 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thockin thockin thockin left review comments

@lala123912 lala123912 lala123912 left review comments

@swetharepakula swetharepakula swetharepakula approved these changes

@aojea aojea Awaiting requested review from aojea

Assignees

@thockin thockin

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/network Categorizes an issue or PR as relevant to SIG Network. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
v1.21
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@robscott @k8s-ci-robot @fejta-bot @swetharepakula @thockin @lala123912