Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Job CRD status not updated upon job completion #1478

Closed
1 of 2 tasks
z103cb opened this issue Oct 11, 2023 · 1 comment · Fixed by #1539
Closed
1 of 2 tasks

[Bug] Job CRD status not updated upon job completion #1478

z103cb opened this issue Oct 11, 2023 · 1 comment · Fixed by #1539
Assignees
Labels
bug Something isn't working rayjob

Comments

@z103cb
Copy link
Contributor

z103cb commented Oct 11, 2023

Search before asking

  • I searched the issues and found no similar issues.

KubeRay Component

ray-operator

What happened + What you expected to happen

When submitting a KubeRay job (ray job CRD) to be executed on a previously provisioned cluster, the ray job status is not set to v1alpha1.JobStatusSucceeded.

The underlying batchv1.Job status is completed.

Reproduction script

To reproduce the issue:

  1. Create k8s cluster
  2. Deploy operator (v1.0.0-rc.0+) in default namespace
  3. Create ray-system namespace: kubectl create namespace ray-system
  4. Create a KubeRay cluster: kubectl apply -f cluster.yaml.txt
  5. Wait for cluster to be up an running
  6. Create a KubeRay Job: kubectl apply -f job.yaml.txt
  7. Watch the job which doesn't have its status updated.

The job batchv1.Job has a status of

status:
  completionTime: "2023-10-11T06:56:19Z"
  conditions:
  - lastProbeTime: "2023-10-11T06:56:19Z"
    lastTransitionTime: "2023-10-11T06:56:19Z"
    status: "True"
    type: Complete
  startTime: "2023-10-11T06:55:46Z"
  succeeded: 1

Also, deleting the ray job doesn't delete the associated batchv1.Job

kubectl delete rayjob -n ray-system sample-job
# rayjob.ray.io "sample-job" deleted
kubectl get job -n ray-system sample-job

# NAME         COMPLETIONS   DURATION   AGE
# sample-job   1/1           33s        28m

Anything else

Fairly consistent.

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!
@astefanutti
Copy link
Contributor

@kevin85421 could you please assign it to me?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working rayjob
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants