kubeval doesn't stop if job restartPolicy is wrong

[icy]

We have some Job configuration and we expect kubeval to stop us if developers provide wrong settings. However, kubeval (0.15.0) simply ignored and at the final step we got an error with helm upgrade

Error: UPGRADE FAILED: failed to create resource: Job.batch "foo-tw-232-tre-two-worker-job" is invalid: 
spec.template.spec.restartPolicy: Unsupported value: "Always": supported values: "OnFailure", "Never"
helm.go:84: [debug] Job.batch "foo-tw-232-tre-two-worker-job" is invalid: 
    spec.template.spec.restartPolicy: Unsupported value: "Always": supported values: "OnFailure", "Never"
failed to create resource
helm.sh/helm/v3/pkg/kube.(*Client).Update.func1
    /home/circleci/helm.sh/helm/pkg/kube/client.go:190
helm.sh/helm/v3/pkg/kube.ResourceList.Visit
    /home/circleci/helm.sh/helm/pkg/kube/resour

[yannh]

Hi @icy , I would point to my answer here #257 - Kubeval only validates Kubernetes manifests following the Swagger file released by the k8s project. This is not the same validation that kubectl, or the Kubernetes server, would do. In this case it doesnt seem like the list of supported values is a constraint in the swagger file.

[icy]

Hi @icy , I would point to my answer here #257 - Kubeval only validates Kubernetes manifests following the Swagger file released by the k8s project. This is not the same validation that kubectl, or the Kubernetes server, would do. In this case it doesnt seem like the list of supported values is a constraint in the swagger file.

I see. The swagger doesn't provide rich information (e.g, normal Pod can have Always but job doesn't have that value.)

Do you think there is any plug-gable way that allows us to have some kustomization checks other than default kubeval checks?

[yannh]

Not sure if that is what you re looking for, but from the same author as kubeval: https://github.com/open-policy-agent/conftest
I guess in some cases, actually changing the swagger definition to get it to enforce the list of valid values could help too :)

[icy]

Not sure if that is what you re looking for, but from the same author as kubeval: https://github.com/open-policy-agent/conftest
I guess in some cases, actually changing the swagger definition to get it to enforce the list of valid values could help too :)

Thanks for the link. We haven't had a chance to implement any OPA-related thing yet, though OPA can help to solve many problems.

That's said, implementing our own validations/tests for k8s-server constraints looks like a reinvent-the-wheel job doesn't it?

[yannh]

it does... I don't know exactly how the Kubernetes Swagger file is generated, but I guess that would be a good place to fix this :)

[yannh]

I have opened kubernetes/kube-openapi#218 ! I m playing with it but it's not the easiest thing in the world ;)

[icy]

I have opened kubernetes/kube-openapi#218 ! I m playing with it but it's not the easiest thing in the world ;)

Thanks a lot for your time. I totally agree that's not the easiest thing ;)

I see that restartPolicy accepts different set of values. For job there are only two (https://kubernetes.io/docs/concepts/workloads/controllers/job/#pod-template)

Only a RestartPolicy equal to Never or OnFailure is allowed.

I don't now how that ends up; you're right the openApi is quite confusing.