Write to mlpipeline-ui-metadata as non-root user?

[haudren]

What happened:
I have a container that runs a pipeline step as non root. I wanted to write to /mlpipeline-ui-metadata.json but got a permission denied. I tried to create it as root and give permissions to my user to write to it but, this results in a blank output.

Note: this is Kubeflow 0.5.0.

What did you expect to happen:
Given that mlpipeine-ui-metadata.json is an output mount, it should be writable by any user. If not, then modifying it should trigger a UI refresh.

What steps did you take:
First scenario:

• Create a pipeline from a non-root container (USER user in the Dockerfile)
• Try to write to /mlpipeline-ui-metadata.json
• Permission denied

Second scenario:

• Create a pipeline from a this Dockerfile:

FROM python:3
RUN touch /mlpipeline-ui-metadata.json
RUN chmod 777 /mlpipeline-ui-metadata.json
RUN useradd -ms /bin/bash my_user
USER my_user
/* Put here the code that writes to metadata file */

• Metadata is written, but doesn't seem to show up in the UI (Run output is empty, step's artifacts seem to spin indefinitely).

[haudren]

My bad using option 2 (creating the file as root in the container, then changing permissions), I was able to render the basic markdown. However it seems like my custom web-app is not showing up. I'll investigate, but it's probably a path issue of sorts.

[haudren]

I figured out the issue, it seems like the web-app is trying to access the file at the original location inside the container:
Error: Unsupported storage path: /home/user/test.html WorkflowParser.ts:225:12

I am trying to use the web-app display type.

[Ark-kun]

Workaround:
Write the file to any location (e.g. /tmp/mlpipeine-ui-metadata.json). Set the artifact path when creating the ContainerOp:
(? Do you use ContainerOp when wrapping your container?)

ContainerOp(
   ...,
   output_artifact_paths={
     'mlpipeline-ui-metadata': '/tmp/outputs/mlpipeline-ui-metadata.json',
     'mlpipeline-metrics': '/tmp/outputs/mlpipeline-metrics.json',
   },

[haudren]

Yes I do use ContainerOp, your fix sounds much better than what my workaround. I see the doc is outdated on the website, I can try to make a PR later this week maybe. Do you have some pointers to what should be updated?

Also for my second question, is there any support for kubeflow itself storing the artifacts? I'd like to not have to care where I store those results files (we are on-prem) as they are only ever used for visualization in the UI.

[Ark-kun]

Also for my second question, is there any support for kubeflow itself storing the artifacts?

I would really like to make this happen. Please add a feature request to prioritize this feature or +1 #336

[haudren]

Thanks a lot for the hard work, using ouput_artifact_paths works great.

[Ark-kun]

@haudren Please migrate from ouput_artifact_paths to file_outputs which now also works for artifacts.

[haudren]

Thanks a lot! I guess I need the latest kfp for this, do I also need the latest Kubeflow deployment (0.7.0)?

Last question, it says that file_outputs supports large files, but does it support directories as well?

[Ark-kun]

Thanks a lot! I guess I need the latest kfp for this, do I also need the latest Kubeflow deployment (0.7.0)?

You only need the latest SDK.

Last question, it says that file_outputs supports large files, but does it support directories as well?

Good question. It does support directories. That name was a bad choice.

BTW, check some modern samples, especially "104 - Passing data for python components" and "106 - Creating components from command-line program". They demonstrate big data passing without manually dealing with ContainerOp.