cmd/kube-scheduler: return error instead of os.Exit when something goes wrong #104503

sanposhiho · 2021-08-22T10:28:36Z

Hello team.

What type of PR is this?

/kind bug

What this PR does / why we need it:

The defer function will not be called if os.Exit.
https://play.golang.org/p/f7IjXeb62dd

deferred functions are not run.
https://golang.org/pkg/os/#Exit

Which issue(s) this PR fixes:

part of #102231

Special notes for your reviewer:

Does this PR introduce a user-facing change?

kube-scheduler now doesn't print any usage message when unknown flag is specified

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

k8s-ci-robot · 2021-08-22T10:28:43Z

@sanposhiho: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot · 2021-08-22T10:28:44Z

Hi @sanposhiho. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ash2k · 2021-08-22T10:49:13Z

cmd/kube-scheduler/app/server.go

@@ -79,15 +79,14 @@ suitable Node. Multiple different schedulers may be used within a cluster;
 kube-scheduler is the reference implementation.
 See [scheduling](https://kubernetes.io/docs/concepts/scheduling-eviction/)
 for more information about scheduling and the kube-scheduler component.`,
-		Run: func(cmd *cobra.Command, args []string) {
+		RunE: func(cmd *cobra.Command, args []string) error {


I suggest to not change the error messages. Just return the errors to preserve the current behavior, whatever it is.

Fixed thanks

You may also want to set SilenceErrors and SilenceUsage to avoid Cobra printing the error and the usage information on the error.

Ok, fixed.
As you said, if we don't set SilenceErrors/SilenceUsage, it seems that cobra changes error msg and adds usage message on error msg. thanks

https://github.com/spf13/cobra/blob/v1.2.1/command.go#L975-L994

ash2k · 2021-08-22T13:24:23Z

/ok-to-test

sanposhiho · 2021-08-22T13:48:02Z

/retest

sanposhiho · 2021-08-22T14:48:39Z

/retest

chendave

/lgtm

ahg-g · 2021-08-26T14:39:17Z

Thanks! did you manually try to test the changes? can you show us the difference between the old and new behavior?

sanposhiho · 2021-08-26T15:42:18Z

Hello @ahg-g.

I've tried it briefly

1. When cobra.Command.RunE return an error.

(For this case, I made RunE always return an error(something goes wrong).)

 $ ./_output/bin/kube-scheduler              
something goes wrong

This is the same output as old. (but, internally, logs.FlushLogs() will be executed only on this PR code)

2. When user specify wrong flag

$ ./_output/bin/kube-scheduler --add-dir-hea
unknown flag: --add-dir-hea

This output is different from old (see below). This is because we set SilenceErrors and SilenceUsage true.

# on release-1.22
$ ./_output/bin/kube-scheduler --add-dir-hea
Error: unknown flag: --add-dir-hea
Usage:
  kube-scheduler [flags]

Misc flags:

      --config string                                                                                                                                                                                            
                The path to the configuration file. The following flags can overwrite fields in this file:
                  --policy-config-file
                  --policy-configmap
                  --policy-configmap-namespace

.....(super long usage message)

But if we don't set SilenceErrors and SilenceUsage on this PR, usage will be print when cobra.Command.RunE return an error. (And this is different output from old)

# not set SilenceErrors and SilenceUsage 
$ ./_output/bin/kube-scheduler     
Error: something goes wrong
Usage:
  kube-scheduler [flags]

Misc flags:

      --config string                                                                                                                                                                                            
                The path to the configuration file. The following flags can overwrite fields in this file:
                  --policy-config-file
                  --policy-configmap
                  --policy-configmap-namespace

.....(super long usage message)

ahg-g · 2021-08-26T15:57:24Z

Thanks!

/lgtm
/approve

k8s-ci-robot · 2021-08-26T15:57:56Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahg-g, sanposhiho

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~cmd/kube-scheduler/OWNERS~~ [ahg-g]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ahg-g

/hold

ahg-g · 2021-08-26T15:58:19Z

cmd/kube-scheduler/app/server.go

 		},
+		SilenceErrors: true,


what does this mean exactly?

When SilenceErrors = false (or not set), an error will be printed with the prefix Error:. In the above example, it prints Error: something goes wrong.
We print error on this line, so I think we don't need this.

https://github.com/spf13/cobra/blob/v1.2.1/command.go#L983-L987

So, if we set false (or don't set SilenceErrors), error is printed twice like this.

$ ./_output/bin/kube-scheduler Error: something goes wrong something goes wrong

thanks for the explanation

I'm going to replace this with a more generic solution that'll work for most of the Kubernetes commands in #105076.

See #105076 (comment)

It would be good to get some additional eyes on that. Perhaps someone can even try out the PR?

I ran tests manually with kube-scheduler as example and I think it still works as you intended here.

ahg-g · 2021-08-26T16:17:06Z

/hold cancel

sanposhiho · 2021-08-26T16:17:32Z

This output is different from old (see below). This is because we set SilenceErrors and SilenceUsage true.

Should we add this change to the release notes? (Like kube-scheduler now don't print any usage message when unknown flag is specified)

ahg-g · 2021-08-26T16:19:07Z

sure, wont hurt to do that.

s/don't/doesn't

Fix: return error instead of os.Exit when something goes wrong

b8ae016

k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Aug 22, 2021

k8s-ci-robot requested review from ahg-g and chendave August 22, 2021 10:29

ash2k reviewed Aug 22, 2021

View reviewed changes

Fix: not change the error messages

a06d613

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 22, 2021

sanposhiho mentioned this pull request Aug 22, 2021

replace klog.Fatal and os.Exit for defer funcs #102231

Open

Fix: set SilenceErrors and SilenceUsage not to change error message

0252865

chendave reviewed Aug 26, 2021

View reviewed changes

k8s-ci-robot assigned chendave Aug 26, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 26, 2021

k8s-ci-robot assigned ahg-g Aug 26, 2021

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 26, 2021

ahg-g reviewed Aug 26, 2021

View reviewed changes

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 26, 2021

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 26, 2021

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Aug 26, 2021

k8s-ci-robot merged commit d0f6983 into kubernetes:master Aug 26, 2021

k8s-ci-robot added this to the v1.23 milestone Aug 26, 2021

sanposhiho mentioned this pull request Sep 6, 2021

Fix(kube-proxy): remove os.Exit from cobra.Command #104750

Merged

This was referenced Sep 21, 2021

cmd/kube-scheduler: add usage message only when parsing flags failed #105175

Merged

fix flushlogs not working together with exit in staging repos #104776

Closed

lonelyCZ mentioned this pull request Jan 20, 2022

os.Exit will not wait for the defer function karmada-io/karmada#1289

Merged

hanweisen mentioned this pull request Jun 13, 2022

bug fix: os.exit will not wait for the defer function clusterpedia-io/clusterpedia#225

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/kube-scheduler: return error instead of os.Exit when something goes wrong #104503

cmd/kube-scheduler: return error instead of os.Exit when something goes wrong #104503

sanposhiho commented Aug 22, 2021 •

edited

k8s-ci-robot commented Aug 22, 2021

k8s-ci-robot commented Aug 22, 2021

ash2k Aug 22, 2021

sanposhiho Aug 22, 2021

ash2k Aug 22, 2021

sanposhiho Aug 23, 2021

ash2k commented Aug 22, 2021

sanposhiho commented Aug 22, 2021

sanposhiho commented Aug 22, 2021

chendave left a comment

ahg-g commented Aug 26, 2021

sanposhiho commented Aug 26, 2021 •

edited

ahg-g commented Aug 26, 2021

k8s-ci-robot commented Aug 26, 2021

ahg-g left a comment

ahg-g Aug 26, 2021

sanposhiho Aug 26, 2021 •

edited

sanposhiho Aug 26, 2021 •

edited

ahg-g Aug 26, 2021

pohly Sep 29, 2021

ahg-g commented Aug 26, 2021

sanposhiho commented Aug 26, 2021 •

edited

ahg-g commented Aug 26, 2021

cmd/kube-scheduler: return error instead of os.Exit when something goes wrong #104503

cmd/kube-scheduler: return error instead of os.Exit when something goes wrong #104503

Conversation

sanposhiho commented Aug 22, 2021 • edited

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

k8s-ci-robot commented Aug 22, 2021

k8s-ci-robot commented Aug 22, 2021

ash2k Aug 22, 2021

Choose a reason for hiding this comment

sanposhiho Aug 22, 2021

Choose a reason for hiding this comment

ash2k Aug 22, 2021

Choose a reason for hiding this comment

sanposhiho Aug 23, 2021

Choose a reason for hiding this comment

ash2k commented Aug 22, 2021

sanposhiho commented Aug 22, 2021

sanposhiho commented Aug 22, 2021

chendave left a comment

Choose a reason for hiding this comment

ahg-g commented Aug 26, 2021

sanposhiho commented Aug 26, 2021 • edited

1. When cobra.Command.RunE return an error.

2. When user specify wrong flag

ahg-g commented Aug 26, 2021

k8s-ci-robot commented Aug 26, 2021

ahg-g left a comment

Choose a reason for hiding this comment

ahg-g Aug 26, 2021

Choose a reason for hiding this comment

sanposhiho Aug 26, 2021 • edited

Choose a reason for hiding this comment

sanposhiho Aug 26, 2021 • edited

Choose a reason for hiding this comment

ahg-g Aug 26, 2021

Choose a reason for hiding this comment

pohly Sep 29, 2021

Choose a reason for hiding this comment

ahg-g commented Aug 26, 2021

sanposhiho commented Aug 26, 2021 • edited

ahg-g commented Aug 26, 2021

sanposhiho commented Aug 22, 2021 •

edited

sanposhiho commented Aug 26, 2021 •

edited

sanposhiho Aug 26, 2021 •

edited

sanposhiho Aug 26, 2021 •

edited

sanposhiho commented Aug 26, 2021 •

edited