dockershim: fix started and finished timestamp of the container status #99585

Iceber · 2021-03-01T12:42:41Z

Signed-off-by: Iceber Gu wei.cai-nat@daocloud.io

What type of PR is this?

/kind bug

What this PR does / why we need it:

When use crictl，Finished time is always 1754-08-31T06:49:24.128654848+08:05 in the run and create states, because FinishedAt is -6795364578871345152

[root@iceber-3 ~]# crictl inspect 587444fc34ec9
{
  "status": {
    "id": "587444fc34ec9846c163d0c74bbf985ac73662817a89e16b7328b955a03ef902",
    "metadata": {
      "attempt": 6,
      "name": "dx-arch-stolon-sentinel"
    },
    "state": "CONTAINER_RUNNING",
    "createdAt": "2021-03-01T16:45:23.390180141+08:00",
    "startedAt": "2021-03-01T16:45:23.544299436+08:00",
    "finishedAt": "1754-08-31T06:49:24.128654848+08:05",
    "exitCode": 0,

I think the default started and finished timestamp should be 0(zero value), not -6795364578871345152.

Which issue(s) this PR fixes:

Special notes for your reviewer:

fix crictl: kubernetes-sigs/cri-tools#724

Convert runtimeapi.ContainerStatus to kubecontainer.Status in the toKubeContainerStatus

kubernetes/pkg/kubelet/kuberuntime/kuberuntime_container.go

Lines 510 to 522 in 793390e

    
           	if status.State != runtimeapi.ContainerState_CONTAINER_CREATED { 
        
           		// If container is not in the created state, we have tried and 
        
           		// started the container. Set the StartedAt time. 
        
           		cStatus.StartedAt = time.Unix(0, status.StartedAt) 
        
           	} 
        
           	if status.State == runtimeapi.ContainerState_CONTAINER_EXITED { 
        
           		cStatus.Reason = status.Reason 
        
           		cStatus.Message = status.Message 
        
           		cStatus.ExitCode = int(status.ExitCode) 
        
           		cStatus.FinishedAt = time.Unix(0, status.FinishedAt) 
        
           	} 
        
           	return cStatus 
        
           }

Does this PR introduce a user-facing change?

Return zero time (midnight on Jan. 1, 1970) instead of negative number when reporting startedAt and finishedAt of the not started or a running Pod when using dockershim as a runtime.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

k8s-ci-robot · 2021-03-01T12:42:50Z

Welcome @Iceber!

It looks like this is your first PR to kubernetes/kubernetes 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/kubernetes has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2021-03-01T12:42:51Z

Hi @Iceber. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

pkg/kubelet/dockershim/docker_container.go

pacoxu · 2021-03-01T12:49:50Z

/ok-to-test

Add a release note like below. You may change the word.

fix the wrong FinishedAt of container status

Iceber · 2021-03-01T13:16:40Z

I am going to modify the pull-kubernetes-bazel-test

Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>

ehashman · 2021-03-02T23:34:32Z

/priority backlog
/assign @SergeyKanzhelev

pacoxu · 2021-03-04T09:41:31Z

LGTM
leave to sig-node review

SergeyKanzhelev · 2021-03-04T21:01:38Z

pkg/kubelet/dockershim/docker_container.go

 	var state runtimeapi.ContainerState
 	var reason, message string
+	ct, st, ft := createdAt.UnixNano(), int64(0), int64(0)


would the better fix be to change the defaults in getContainerTimestamps? This way if docker returns bad times, kubernetes will report the same

hm, never mind. I got it. I wonder what docker returns as a Finish time that is converted to -6795364578871345152?

docker returns as a Finished time that is converted to 0(0001-01-01T00:00:00Z)

SergeyKanzhelev · 2021-03-04T21:13:51Z

Please change the release note to:

Return zero time (midnight on Jan. 1, 1970) instead of negative number when reporting startedAt and finishedAt of the not started or a running Pod when using dockershim as a runtime.

SergeyKanzhelev · 2021-03-04T21:14:23Z

/lgtm
/hold

hold for changing the release note

SergeyKanzhelev · 2021-03-05T01:41:15Z

/hold cancel

Iceber · 2021-03-05T01:41:54Z

@SergeyKanzhelev the release note has been changed
PTAL, Thanks

mrunalp · 2021-03-09T00:14:14Z

/approve

k8s-ci-robot · 2021-03-09T00:14:43Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Iceber, mrunalp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~pkg/kubelet/OWNERS~~ [mrunalp]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Iceber · 2021-03-09T01:44:53Z

/test pull-kubernetes-e2e-kind

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Mar 1, 2021

k8s-ci-robot added area/kubelet sig/node Categorizes an issue or PR as relevant to SIG Node. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Mar 1, 2021

k8s-ci-robot requested review from yifan-gu and yujuhong March 1, 2021 12:43

pacoxu reviewed Mar 1, 2021

View reviewed changes

pkg/kubelet/dockershim/docker_container.go Outdated Show resolved Hide resolved

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 1, 2021

Iceber changed the title ~~dockershim: fix FinishedAt of the container status~~ [WIP] dockershim: fix FinishedAt of the container status Mar 1, 2021

Iceber force-pushed the fix-finished-at branch from a352e54 to 1c789eb Compare March 1, 2021 13:53

k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Mar 1, 2021

Iceber changed the title ~~[WIP] dockershim: fix FinishedAt of the container status~~ dockershim: fix started and finished timestamp of the container status Mar 1, 2021

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 1, 2021

dockershim: fix started and finished timestamp of the container status

aff9df2

Signed-off-by: Iceber Gu <wei.cai-nat@daocloud.io>

k8s-ci-robot removed the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label Mar 2, 2021

Iceber mentioned this pull request Mar 2, 2021

fix StartedAt and FinishedAt of the container status kubernetes-sigs/cri-tools#724

Merged

ehashman added this to Triage in SIG Node PR Triage Mar 2, 2021

k8s-ci-robot assigned SergeyKanzhelev Mar 2, 2021

k8s-ci-robot added priority/backlog Higher priority than priority/awaiting-more-evidence. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Mar 2, 2021

ehashman moved this from Triage to Needs Reviewer in SIG Node PR Triage Mar 2, 2021

Iceber mentioned this pull request Mar 3, 2021

cri: fix container status containerd/containerd#5117

Merged

SergeyKanzhelev reviewed Mar 4, 2021

View reviewed changes

k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Mar 4, 2021

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 5, 2021

SergeyKanzhelev moved this from Needs Reviewer to Needs Approver in SIG Node PR Triage Mar 5, 2021

mrunalp moved this from Needs Approver to Done in SIG Node PR Triage Mar 9, 2021

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 9, 2021

k8s-ci-robot merged commit 40d8aed into kubernetes:master Mar 9, 2021

k8s-ci-robot added this to the v1.21 milestone Mar 9, 2021

Iceber deleted the fix-finished-at branch March 12, 2021 09:40

Iceber mentioned this pull request Apr 21, 2021

REQUEST: New membership for Iceber kubernetes/org#2650

Closed

6 tasks

pacoxu mentioned this pull request Aug 6, 2021

nominate pacoxu as sig node reviewer #104186

Merged

13 tasks

AxeZhan mentioned this pull request Sep 3, 2023

Job Failed StartTime is 1970? #120331

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dockershim: fix started and finished timestamp of the container status #99585

dockershim: fix started and finished timestamp of the container status #99585

Iceber commented Mar 1, 2021 •

edited

k8s-ci-robot commented Mar 1, 2021

k8s-ci-robot commented Mar 1, 2021

pacoxu commented Mar 1, 2021

Iceber commented Mar 1, 2021

ehashman commented Mar 2, 2021

pacoxu commented Mar 4, 2021

SergeyKanzhelev Mar 4, 2021

SergeyKanzhelev Mar 4, 2021

Iceber Mar 5, 2021

SergeyKanzhelev commented Mar 4, 2021

SergeyKanzhelev commented Mar 4, 2021

SergeyKanzhelev commented Mar 5, 2021

Iceber commented Mar 5, 2021

mrunalp commented Mar 9, 2021

k8s-ci-robot commented Mar 9, 2021

Iceber commented Mar 9, 2021

	if status.State != runtimeapi.ContainerState_CONTAINER_CREATED {
	// If container is not in the created state, we have tried and
	// started the container. Set the StartedAt time.
	cStatus.StartedAt = time.Unix(0, status.StartedAt)
	}
	if status.State == runtimeapi.ContainerState_CONTAINER_EXITED {
	cStatus.Reason = status.Reason
	cStatus.Message = status.Message
	cStatus.ExitCode = int(status.ExitCode)
	cStatus.FinishedAt = time.Unix(0, status.FinishedAt)
	}
	return cStatus
	}

dockershim: fix started and finished timestamp of the container status #99585

dockershim: fix started and finished timestamp of the container status #99585

Conversation

Iceber commented Mar 1, 2021 • edited

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

k8s-ci-robot commented Mar 1, 2021

k8s-ci-robot commented Mar 1, 2021

pacoxu commented Mar 1, 2021

Iceber commented Mar 1, 2021

ehashman commented Mar 2, 2021

pacoxu commented Mar 4, 2021

SergeyKanzhelev Mar 4, 2021

Choose a reason for hiding this comment

SergeyKanzhelev Mar 4, 2021

Choose a reason for hiding this comment

Iceber Mar 5, 2021

Choose a reason for hiding this comment

SergeyKanzhelev commented Mar 4, 2021

SergeyKanzhelev commented Mar 4, 2021

SergeyKanzhelev commented Mar 5, 2021

Iceber commented Mar 5, 2021

mrunalp commented Mar 9, 2021

k8s-ci-robot commented Mar 9, 2021

Iceber commented Mar 9, 2021

Iceber commented Mar 1, 2021 •

edited