v1beta2 wishlist

[alculquicondor]

What would you like to be cleaned:

In a v1beta2 API, we would like to make the following breaking changes:

• Workload
  • Report resource usage by pod, instead of the entire podset, in the admission struct.
  • Remove 'kueue.x-k8s.io/original-node-selectors' annotation
• LocalQueue
  • Rename status.flavorUsage to status.flavorsUsage.

Why is this needed:

As we add more features, some field didn't age well.

[tenzen-y]

Remove 'kueue.x-k8s.io/original-node-selectors' annotation from workload

ref: #771

[lowang-bh]

Has been removed in #834

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[tenzen-y]

/remove-lifecycle stale

[PBundyra]

Remove .spec.AdmissionChecks field in ClusterQueue and convert its content to .spec.AdmissionCheckStrategy
KEP: #1935
Issue: #1432

[IrvingMg]

Change .spec.flavorFungibility, .spec.preemption, .spec.preemption.borrowWithinCohort type from pointer to value for ClusterQueueSpec.
Discussion: #1972 (comment)

[mimowo]

Maybe we can consider to move the workload's spec.active to status.
Currently, the workload_controller needs write access to spec, which isn't recommended (see API conventions). Also, it makes tricky to set the reason for workload's deactivation reliably, see thread.

[alculquicondor]

I'm not too sure about that one. active expresses a desired state, as such, it should be in the spec. It can come from the user or it can come from the system.

[tenzen-y]

I'm not too sure about that one. active expresses a desired state, as such, it should be in the spec. It can come from the user or it can come from the system.

That is a good point.
In my point, I'm wondering if we should define the dedicated API for the kueue-controller-manager to deactivate the Workload in the status field.

[alculquicondor]

Tricky... because we also want the users to be able to reset the active field back to true. And they definitely shouldn't have access to status.

[PBundyra]

Deprecate the QueueVisibility API.
Issue: #2256

[tenzen-y]

Tricky... because we also want the users to be able to reset the active field back to true. And they definitely shouldn't have access to status.

Yeah, I agree with you. My motivation is mitigating compexity of mechanism to deactivate workload by exceeding the backoffLimit in the workload controller.

Once we move or add active field in the Workload, that allows us to deactivate a workload by a single API call.

Current behavior needs to 3 reconciling: spec.active=false (workload controller) -> add Evicted condition to a workload (workload controller) -> stop Job (jobframework reconciler).
New .status.active needs to 2 reconciling: add Evicted condition to a workload (workload controller) -> stop Job (jobframework reconciler).

But, I'm sure your concerns. So, let us seek any alternative approach to mitigate complexity of mechanism of deactivation.

[PBundyra]

Deprecate or remove the retryDelayMinutes from the AdmissionCheck API
Issue: #2437

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[mimowo]

/remove-lifecycle stale

[mimowo]

We may also revisit promoting domains for some labels (for example to support kueue.kubernetes.io/queue-name along with kueue.x-k8s.io/queue-name), see #2858

[tenzen-y]

We may also revisit promoting domains for some labels (for example to support kueue.kubernetes.io/queue-name along with kueue.x-k8s.io/queue-name), see #2858

AFAIK, @kerthcet tried to add queueName field to the core PodSpec API. We may be able to retry it before adding the kueue.kuebernetes.io/queue-name. Although I'm not sure if the label is acceptable by core kube API.

[mimowo]

Also adding status to ResourceFlavor could be useful, especially for TAS, currently we only have Spec:

kueue/apis/kueue/v1beta1/resourceflavor_types.go

Line 35 in cdbb7a1

Spec ResourceFlavorSpec `json:"spec,omitempty"`

[alculquicondor]

That doesn't need a new API version

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[mimowo]

/remove-lifecycle stale

[tenzen-y]

Fix a field name typo in the Workload resource found at #4297.

incorrect: .status.accumulatedPastExexcutionTimeSeconds.
correct: .status.accumulatedPastExecutionTimeSeconds

kueue/apis/kueue/v1beta1/workload_types.go

Lines 350 to 354 in 4f01318

	// accumulatedPastExexcutionTimeSeconds holds the total time, in seconds, the workload spent
	// in Admitted state, in the previous `Admit` - `Evict` cycles.
	//
	// +optional
	AccumulatedPastExexcutionTimeSeconds *int32 `json:"accumulatedPastExexcutionTimeSeconds,omitempty"`

[gabesaba]

Unify name of Cohort and CQ's parent reference. Perhaps both being parentCohort

See #4473