Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

Commit

Permalink
Docs modification (admin) (#5299)
Browse files Browse the repository at this point in the history
* Readme.md and some grammarly errors due to proper nouns

* Some missing grammarly errors due to proper nouns

* Some missing grammarly errors
  • Loading branch information
Starmys authored Feb 19, 2021
1 parent f652c99 commit ce37841
Show file tree
Hide file tree
Showing 20 changed files with 162 additions and 159 deletions.
18 changes: 9 additions & 9 deletions docs/manual/cluster-admin/basic-management-operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

## Management on Webportal

The webportal provides some basic administration functions. If you log in to it as an administrator, you will find several buttons about administration on the left sidebar, as shown in the following image.
The webportal provides some basic administration functions. If you log in to it as an administrator, you will find several buttons about the administration on the left sidebar, as shown in the following image.

<img src="./imgs/administration.png" width="100%" height="100%" />

Most of these functions are easy to understand. We will go through them quickly in this section.

### Hardware Utilization Page

The hardware page shows the CPU, GPU, memory, disk, network utilization of each node in your cluster. The utilization is shown in different color basically. If you hover your mouse on these colored circles, exact utilization percentage will be shown.
The hardware page shows the CPU, GPU, memory, disk, network utilization of each node in your cluster. The utilization is shown in different colors. If you hover your mouse on these colored circles, the exact utilization percentage will be shown.

<img src="./imgs/hardware.png" width="100%" height="100%" />

Expand All @@ -36,7 +36,7 @@ On the homepage, there is an `abnormal jobs` section for administrators. A job i

### Access Kubernetes Dashboard

There is a shortcut to k8s dashboard on the webportal. However, it needs special authentication for security issues.
There is a shortcut to the k8s dashboard on the webportal. However, it needs special authentication for security issues.

<img src="./imgs/k8s-dashboard.png" width="100%" height="100%" />

Expand Down Expand Up @@ -67,11 +67,11 @@ subjects:
**Step 2.** Run `kubectl apply -f admin-user.yaml`

**Step 3.** Run `kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')`. It will print the token which can be used to login k8s-dashboard.
**Step 3.** Run `kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')`. It will print the token which can be used to login to k8s-dashboard.

## PAI Service Management and Paictl

Generally speaking, PAI services are daemon sets, deployments or stateful sets created by PAI system, running on Kubernetes. You can find them on the [k8s dashboard](#access-kubernetes-dashboard) and [services page](#services-page). For example, `webportal` is a PAI service which provides front-end interface, and `rest-server` is another one for back-end APIs. These services are all configurable. If you have followed the [installation-guide](./installation-guide.md), you can find two files, `layout.yaml` and `services-configuration.yaml`, in folder `~/pai-deploy/cluster-cfg` on the dev box machine. These two files are the default service configuration.
Generally speaking, PAI services are daemon sets, deployments, or stateful sets created by PAI system, running on Kubernetes. You can find them on the [k8s dashboard](#access-kubernetes-dashboard) and [services page](#services-page). For example, `webportal` is a PAI service which provides front-end interface, and `rest-server` is another one for back-end APIs. These services are all configurable. If you have followed the [installation-guide](./installation-guide.md), you can find two files, `layout.yaml` and `services-configuration.yaml`, in folder `~/pai-deploy/cluster-cfg` on the dev box machine. These two files are the default service configuration.

`paictl` is a CLI tool which helps you manage cluster configuration and PAI services. To use it, we recommend you to leverage our dev box docker image to avoid environment-related problems. First, go to the dev box machine, launch the dev box docker by:

Expand Down Expand Up @@ -103,7 +103,7 @@ mkdir -p ~/.kube
vim ~/.kube/config
```

Go to folder `/pai`, try to retrieve your cluster id:
Go to folder `/pai`, try to retrieve your cluster-id:

```bash
cd /pai
Expand All @@ -120,7 +120,7 @@ Here are some basic usage examples of `paictl`:

# pull service config to a certain folder
# the configuration containers two files: layout.yaml and services-configuration.yaml
# if <config-folder> already has these files, they will be overrided
# if <config-folder> already has these files, they will be overridden
./paictl.py config pull -o <config-folder>

# push service config to the cluster
Expand All @@ -142,7 +142,7 @@ Here are some basic usage examples of `paictl`:

If you want to change configuration of some services, please follow the steps of `service stop`, `config push` and `service start`.

For example, if you want to customize webportal, you should modify the `webportal` section in `services-configuration.yaml`. Then use the following command to push the configuration and restart webportal:
For example, if you want to customize webportal, you should modify the `webportal` section in `services-configuration.yaml`. Then use the following command to push the configuration and restart the webportal:

```bash
./paictl.py service stop -n webportal
Expand All @@ -158,7 +158,7 @@ Another example is to restart the whole cluster:
./paictl.py service start
```

You can use `exit` to leave the dev-box container, and use `sudo docker exec -it dev-box bash` to re-enter it if you desire so. If you don't need it any more, use `sudo docker stop dev-box` and `sudo docker rm dev-box` to delete the docker container.
You can use `exit` to leave the dev-box container, and use `sudo docker exec -it dev-box bash` to re-enter it if you desire so. If you don't need it anymore, use `sudo docker stop dev-box` and `sudo docker rm dev-box` to delete the docker container.

## How To Set Up HTTPS

Expand Down
4 changes: 2 additions & 2 deletions docs/manual/cluster-admin/how-to-add-and-remove-nodes.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# How to Add and Remove Nodes

OpenPAI doesn't support changing master nodes, thus, only the solution of adding/removing worker nodes is provided. You can add CPU workers, GPU workers, and other computing device (e.g. TPU, NPU) into the cluster.
OpenPAI doesn't support changing master nodes, thus, only the solution of adding/removing worker nodes is provided. You can add CPU workers, GPU workers, and other computing devices (e.g. TPU, NPU) into the cluster.

## How to Add Nodes

Expand Down Expand Up @@ -133,7 +133,7 @@ If you have configured any PV/PVC storage, please confirm the added worker node

## How to Remove Nodes

Please refer to the operation of add nodes. They are very similar.
Please refer to the operation of adding nodes. They are very similar.

To remove nodes from the cluster, there is no need to modify `hosts.yml`.
Go into `~/pai-deploy/kubespray/`, run
Expand Down
14 changes: 7 additions & 7 deletions docs/manual/cluster-admin/how-to-customize-cluster-by-plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Webportal plugin provides a way to add custom web pages to the OpenPAI webportal. It can communicate with other PAI services, like the rest-server. It could provide customized solutions to different requirements.

As an administrator, you can configure the web portal plugins in the `webportal.plugins` field of `services-configuration.yaml` (If you don't know what `services-configuration.yaml` is, please refer to [PAI Service Management and Paictl](./basic-management-operations.md#pai-service-management-and-paictl)):
As an administrator, you can configure the webportal plugins in the `webportal.plugins` field of `services-configuration.yaml` (If you don't know what `services-configuration.yaml` is, please refer to [PAI Service Management and Paictl](./basic-management-operations.md#pai-service-management-and-paictl)):

```yaml
webportal:
Expand All @@ -18,9 +18,9 @@ webportal:
```
- The `title` field is the title of the web portal plugin listed in the menu, it could be customized by administrators for the same plugin with different configurations.
- The `uri` field is the entry file of the web portal plugin, usually previded by the plugin developer. It may be an absolute URL or a root-relative URL, as the different deploy type of the web portal plugin.
- The `config` field is a key-value dictionary to configure the web portal plugin, available configs are listed in web portal plugin's specific document.
- The `title` field is the title of the webportal plugin listed in the menu, it could be customized by administrators for the same plugin with different configurations.
- The `uri` field is the entry file of the webportal plugin, usually provided by the plugin developer. It may be an absolute URL or a root-relative URL, as the different deploy type of the webportal plugin.
- The `config` field is a key-value dictionary to configure the webportal plugin, available configs are listed in the webportal plugin's specific document.

After modifying the configuration, push it to the cluster and restart webportal by:

Expand All @@ -30,12 +30,12 @@ After modifying the configuration, push it to the cluster and restart webportal
./paictl.py service start -n webportal
```

## Deploy Openpaimarketplace as webportal plugin
## Deploy Openpaimarketplace as Webportal Plugin

[Openpaimarketplace](https://github.com/microsoft/openpaimarketplace) is a place which stores examples and job templates of openpai. Users could use openpaimarketplace to share their jobs or run-and-learn others' sharing job.
[Openpaimarketplace](https://github.com/microsoft/openpaimarketplace) is a place that stores examples and job templates of OpenPAI. Users could use openpaimarketplace to share their jobs or run-and-learn others' sharing jobs.

To deploy openpaimarketplace, please refer to [the project doc](https://github.com/microsoft/openpaimarketplace) about how to deploy the marketplace service and webportal plugin.

After deployment, follow the [previous part](#how-to-install-a-webportal-plugin) to change the webportal configuration with marketplace plugin url and restart webportal. Then you could use marketplace from the sidebar.
After deployment, follow the [previous part](#how-to-install-a-webportal-plugin) to change the webportal configuration with marketplace plugin URL and restart webportal. Then you could use marketplace from the sidebar.

<img src="./imgs/marketplace.png" width="100%" height="100%" />
40 changes: 20 additions & 20 deletions docs/manual/cluster-admin/how-to-manage-users-and-groups.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

## Users and Groups in Basic Authentication Mode

OpenPAI is deployed in basic authentication mode by default. Groups in basic authentication mode are bound to virtual clusters (please refer to [how to set up virtual clusters](./how-to-set-up-virtual-clusters.md) to configure virtual clusters). Two groups, `default` and `admingroup` will be created once OpenPAI is deployed. All users belong to `default` group, and have access to the `default` virtual cluster. All administrators belong to `admingroup`, and have access to all virtual clusters. If there is another virtual cluster named `test-vc`, and an administrator grants it to a user, the user will be in group `test-vc` and have access to the corresponding virtual cluster.
OpenPAI is deployed in basic authentication mode by default. Groups in basic authentication mode are bound to virtual clusters (please refer to [how to set up virtual clusters](./how-to-set-up-virtual-clusters.md) to configure virtual clusters). Two groups, `default` and `admingroup` will be created once OpenPAI is deployed. All users belong to `default` group and have access to the `default` virtual cluster. All administrators belong to `admingroup`, and have access to all virtual clusters. If there is another virtual cluster named `test-vc`, and an administrator grants it to a user, the user will be in group `test-vc` and have access to the corresponding virtual cluster.

For example, if you create an admin user [on the webportal](./basic-management-operations.md#user-management), he will be in `default` and `admingroup`. A non-admin user will be only in `default` group once created. If administrator gives the non-admin user access to `new-vc`, he will be in `default` and `new-vc` group.

A user can see his groups in the profile page. First click `View my profile` in the right-top corner.
A user can see his groups on the profile page. First click `View my profile` in the right-top corner.

<img src="./imgs/view-profile.png" width="100%" height="100%" />

Expand All @@ -25,7 +25,7 @@ In this section, we will cover how to set up the integration step by step.

#### Note

Previous user data in webportal is required to be mapping/migrate to AAD. Once the integration is enabled, instead of using basic user authentication, OpenPAI will switch to use (and only use) AAD as user authentication mechanism. To set up AAD, please follow the instructions [here](./basic-management-operations.md#how-to-set-up-https) to set up HTTPS access for OpenPAI first.
Previous user data in webportal is required to be mapping/migrate to AAD. Once the integration is enabled, instead of using basic user authentication, OpenPAI will switch to use (and only use) AAD as the user authentication mechanism. To set up AAD, please follow the instructions [here](./basic-management-operations.md#how-to-set-up-https) to set up HTTPS access for OpenPAI first.


#### [Rest-server] Configuration AAD
Expand Down Expand Up @@ -108,7 +108,7 @@ authentication:

# Admin group name and its user list
admin-group:
# The group named showed in OpenPAI system.
# The group named showed in the OpenPAI system.
groupname: admingroup
description: "admin's group"
# The group alias (groupname) in Azure Active directory
Expand All @@ -117,15 +117,15 @@ authentication:
# Group for default vc.
# For yarn default queue hack.
default-group:
# The group named showed in OpenPAI system.
# The group named showed in the OpenPAI system.
groupname: default
description: "group for default vc"
# The group alias (groupname) in Azure Active directory
externalName: "team_alias_b"

# If you cluster you have configured several yarn vc, except default vc (it has been created in the default-group), you should configure group for each vc in the following list
# If you cluster you have configured several yarn VC, except default VC (it has been created in the default-group), you should configure group for each VC in the following list
grouplist:
# The group named showed in OpenPAI system.
# The group named showed in the OpenPAI system.
- groupname: forexample1
description: forexample1
# The group alias (groupname) in Azure Active directory
Expand All @@ -139,13 +139,13 @@ authentication:
##### Clean Previous Data
Please clean all users' data. Because in this mode, user's permission will be managed by azure active directory. The local data is useless.
Please clean all users' data. Because in this mode, the user's permission will be managed by the Azure active directory. The local data is useless.
```bash
./paictl.py service delete -n rest-server
```

##### After all the steps above, push the configuration, and restart all OpenPAI services.
##### After all the steps above, push the configuration and restart all OpenPAI services.

```bash
./paictl.py service stop
Expand All @@ -157,7 +157,7 @@ Please clean all users' data. Because in this mode, user's permission will be ma

##### Start Service stage

After start rest-server, please ensure that the following task is successfully executed.
After start the rest-server, please ensure that the following task is successfully executed.

- namespace named ```pai-group``` and ```pai-user-v2```are created

Expand All @@ -171,34 +171,34 @@ After start rest-server, please ensure that the following task is successfully e
<img src="./imgs/aad/group-created.png" alt="paictl overview picture" style="float: center; margin-right: 10px;" />
</div>

- Every group have an `acls` in extension field.
- Every group has an `acls` in the extension field.

<div align="center">
<img src="./imgs/aad/admin_group_detail.png" alt="paictl overview picture" style="float: center; margin-right: 10px;" />
<img src="./imgs/aad/default_group_detail.png" alt="paictl overview picture" style="float: center; margin-right: 10px;" />
</div>

- Please Login through OpenPAI's webportal, then please check whether your user's data is created in the secret of ```pai-user-v2``` namespace.
- Please login through OpenPAI's webportal, then please check whether your user's data is created in the secret of ```pai-user-v2``` namespace.

<div align="center">
<img src="./imgs/aad/user_created.png" alt="paictl overview picture" style="float: center; margin-right: 10px;" />
</div>

- please check the created user data. There should be an empty extension and a non-empty grouplist.
- please check the created user data. There should be an empty extension and a non-empty group list.

<div align="center">
<img src="./imgs/aad/user_detail.png" alt="paictl overview picture" style="float: center; margin-right: 10px;" />
</div>

- please submit a test job in default vc, and then submit the same job to another vc.
- please submit a test job in default VC, and then submit the same job to another VC.

- please check whether admin user can access to the administration tab.
- please check whether the admin user can access the administration tab.

<div align="center">
<img src="./imgs/aad/admin_view.png" alt="paictl overview picture" style="float: center; margin-right: 10px;" />
</div>

- please create a vc, then check whether a corresponding group is created.
- please create a VC, then check whether a corresponding group is created.

<div align="center">
<img src="./imgs/aad/add_vc.png" alt="paictl overview picture" style="float: center; margin-right: 10px;" />
Expand All @@ -208,24 +208,24 @@ After start rest-server, please ensure that the following task is successfully e
<img src="./imgs/aad/test_group_detail.png" alt="paictl overview picture" style="float: center; margin-right: 10px;" />
</div>

- After creating the new vc, please check whether the new vc is available for admin at home page.
- After creating the new VC, please check whether the new VC is available for the admin on the home page.

<div align="center">
<img src="./imgs/aad/admin_home.png" alt="paictl overview picture" style="float: center; margin-right: 10px;" />
</div>

- Delete the test vc, then please check whether the corresponding group is deleted.
- Delete the test VC, then please check whether the corresponding group is deleted.

<div align="center">
<img src="./imgs/aad/vc_delete.png" alt="paictl overview picture" style="float: center; margin-right: 10px;" />
</div>

- After deleting the vc, please check whether the group is removed from `pai-group` secrets.
- After deleting the VC, please check whether the group is removed from `pai-group` secrets.

<div align="center">
<img src="./imgs/aad/group_delete.png" alt="paictl overview picture" style="float: center; margin-right: 10px;" />
</div>

##### If test failed

Please try to delete the rest-server, and then try to start it again. If fail again, please provide detail information and create issue ticket in github.
Please try to delete the rest-server, and then try to start it again. If it fails again, please provide detailed information and create an issue ticket in Github.
Loading

0 comments on commit ce37841

Please sign in to comment.