Some problems about Team Storage Detail

[siaimes]

What would you like to be added:

Following the "upload" part of the storage usage tutorial, the user will mount the root of the NFS server to upload files, however, the folder that the user mounts to the container is server://${PAI_USER_NAME} (share: "false" in PVC ).

On the one hand, this exposes all users' files to this user, so that the security of user data cannot be guaranteed. On the other hand, it caused confusion to the user. The file was uploaded clearly, but it could not be found in the container.

My solution is to restrict only the machines in the cluster to mount on the NFS server. When the user needs to upload files, he opens a CPU-only container and then uploads files with the SFTP tool. This can solve the security problem of the NFS server itself, that is, prohibit LAN hackers from mounting the NFS folder. However, it cannot be ruled out that users follow this tutorial to mount the shared folder of the NFS server in the container.

So this page and this page needs to be modified.

Why is this needed:

Without this feature, how does the current module work:

Components that may involve changes:

[siaimes]

It would be better if it could be done technically, but based on my knowledge, this may not be possible.

[Binyang2014]

Thanks for share this practice. Restrict machine to mount NFS is a good idea.
For NFS server, it use uid/gid to control file permission. So if we want to control the file permission. We need to give correct uid/gid to the user job pod. We also need to give correct uid/gid to the restricted mount machine. Then user can only access the file with same uid/gid.

But it's will be a problem if new user added. We need to add the user to both nfs server and pai system. Maybe consider to use cloud storage (AWS S3/Azure blob/Alibaba OSS) is an other solution.

[siaimes]

Thanks for share this practice. Restrict machine to mount NFS is a good idea.
For NFS server, it use uid/gid to control file permission. So if we want to control the file permission. We need to give correct uid/gid to the user job pod. We also need to give correct uid/gid to the restricted mount machine. Then user can only access the file with same uid/gid.

But it's will be a problem if new user added. We need to add the user to both nfs server and pai system. Maybe consider to use cloud storage (AWS S3/Azure blob/Alibaba OSS) is an other solution.

Because the NFS server may not belong to the cluster, it seems that the uid/gid cannot be automatically shared with the NFS server in the background when adding users.

If openpai can disable the mount command in the container, then this problem will be solved.

Due to policy reasons, it is convenient for universities to use scientific research funds to buy equipment, but it is inconvenient to use cloud services. This is why we maintain a cluster, although we are only a small team.

[Binyang2014]

For disable mount commands. I think mount is only allow by root user. If we launch container with non-root user (or user don't have permission to run sudo command), we can solve this problem. But this solution may bring some inconvenient.

For more advantage feature, you can limit the allowed system call in container, your can refer: https://kubernetes.io/docs/tutorials/clusters/seccomp/

[siaimes]

After this PR is released. I can up another Nginx service and use the sub_filter plugin to block the team storage detail page, and my concern will be solved.

[Binyang2014]

Close this, feel free to reopen

[siaimes]

This is my solution for reference to friends in need:

sub_filter 'l.a.createElement(p.a,{variant:"large"},"Upload"),' '';
sub_filter '"nfs"===r.type&&l.a.createElement' '"nfs_delete"===r.type&&l.a.createElement';
sub_filter 'By selecting team storage, the storage server will be automatically mounted to Path when job runs. You could copy/read/write like local folder.' 'By selecting team storage, the storage server will be automatically mounted to Path when job runs. You could SSH into job container to copy/read/write like local folder or upload and download files by using ftp tools.';

[siaimes]

For disable mount commands. I think mount is only allow by root user. If we launch container with non-root user (or user don't have permission to run sudo command), we can solve this problem. But this solution may bring some inconvenient.

For more advantage feature, you can limit the allowed system call in container, your can refer: https://kubernetes.io/docs/tutorials/clusters/seccomp/

Unfortunately, seccomp has been supported since v1.22, while openpai is based on v1.15.