Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

right issue with train_pipeline.py job (chap 2) #140

Open
GodefroyClair opened this issue Apr 3, 2021 · 3 comments
Open

right issue with train_pipeline.py job (chap 2) #140

GodefroyClair opened this issue Apr 3, 2021 · 3 comments

Comments

@GodefroyClair
Copy link

When I try the pipeline of chapter 2 locally with minikube v1.18.1 & Docker version 20.10.5, I have the following error:

serve
simple-sci-kit-kf-pipeline-8jgpg-992827095
This step is in Failed state with this message: Error from server (Forbidden): error when creating "/tmp/manifest.yaml": seldondeployments.machinelearning.seldon.io is forbidden: User "system:serviceaccount:kubeflow:pipeline-runner" cannot create resource "seldondeployments" in API group "machinelearning.seldon.io" in the namespace "kubeflow"
@akhimji
Copy link

akhimji commented Apr 12, 2021

Looks like you are missing the Role and RoleBinding. Look under the folder

intro-to-ml-with-kubeflow-examples/ch2_seldon_examples/

applying the following and then try again. This should give you the required permissions.

pipeline_role.yaml
pipeline_rolebinding.yaml

HTH

@GodefroyClair
Copy link
Author

Thx HTH, this works.

@zinzinhust96
Copy link

@GodefroyClair, I am sorry for bringing up this issue

I have been following the instructions of chapter 2 in the book

First, I ran the step to create the training workflow pipeline. It ran successfully:
dsl-compile --py train_pipeline.py --output job.yaml

After that, I logged in the Kubeflow UI via
kubectl port-forward svc/istio-ingressgateway -n istio-system 7777:80
and then went to localhost:7777.

Next, I need to create a namespace. I tried to type in "kubeflow", as this is the namespace you use to run and encounter the error. But the namespace "kubeflow" is not allowed

Capture

I have tried using other name for the namespace in this step (such as "anonymous"). But I found that in kubectl get namespace, it create a new namespace. I then use the web UI to import job.yaml pipeline and run the experiment, but I encounter the same error as you did (with the new namespace ofc).

This step is in Failed state with this message: Error from server (Forbidden): error when creating "/tmp/manifest.yaml": seldondeployments.machinelearning.seldon.io is forbidden: User "system:serviceaccount:anonymous:pipeline-runner" cannot create resource "seldondeployments" in API group "machinelearning.seldon.io" in the namespace "anonymous"

The solution of @akhimji does not work

Can you guide me how to run the experiments with the "kubeflow" namespace like you did, and what step am I doing wrong?
Thanks a bunch!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants