From 0282b53f621a8d170f8e9ef8891630190ba8a366 Mon Sep 17 00:00:00 2001 From: Alexandra Johnson Date: Mon, 8 Apr 2019 15:44:03 -0700 Subject: [PATCH 1/9] Update REAME example links for v1alpha1 --- README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 81fb383f922..8bccc713914 100644 --- a/README.md +++ b/README.md @@ -78,14 +78,14 @@ Each component communicates with others via GRPC and the API is defined at `pkg/ ## Getting Started -Please see [README.md](./examples/README.md) for more details. +Please see [README.md](./examples/v1alpha1/README.md) for more details. ## Web UI Katib provides a Web UI. You can visualize general trend of Hyper parameter space and each training history. You can use -[random-example](https://github.com/kubeflow/katib/blob/master/examples/random-example.yaml) or -[other examples](https://github.com/kubeflow/katib/blob/master/examples) to generate a similar UI. +[random-example](https://github.com/kubeflow/katib/blob/master/examples/v1alpha1/random-example.yaml) or +[other examples](https://github.com/kubeflow/katib/blob/master/examples/v1alpha1) to generate a similar UI. ![katibui](https://user-images.githubusercontent.com/10014831/48778081-a4388b80-ed17-11e8-938b-fc59a5d2e574.gif) ## API Documentation @@ -189,21 +189,21 @@ spec: If you are not using GKE and you don't have StorageClass for dynamic volume provisioning at your cluster, you have to create pvc and pv ``` -kubectl create -f https://raw.githubusercontent.com/kubeflow/katib/master/examples/tfevent-volume/tfevent-pvc.yaml +kubectl create -f https://raw.githubusercontent.com/kubeflow/katib/master/examples/v1alpha1/tfevent-volume/tfevent-pvc.yaml -kubectl create -f https://raw.githubusercontent.com/kubeflow/katib/master/examples/tfevent-volume/tfevent-pv.yaml +kubectl create -f https://raw.githubusercontent.com/kubeflow/katib/master/examples/v1alpha1/tfevent-volume/tfevent-pv.yaml ``` This is example for tfjob operator ``` -kubectl create -f https://raw.githubusercontent.com/kubeflow/katib/master/examples/tfjob-example.yaml +kubectl create -f https://raw.githubusercontent.com/kubeflow/katib/master/examples/v1alpha1/tfjob-example.yaml ``` This is example for pytorch operator ``` -kubectl create -f https://raw.githubusercontent.com/kubeflow/katib/master/examples/pytorchjob-example.yaml +kubectl create -f https://raw.githubusercontent.com/kubeflow/katib/master/examples/v1alpha1/pytorchjob-example.yaml ``` You can check status of StudyJob From 1e11069c83910c41709f3e0aa2296541ea542634 Mon Sep 17 00:00:00 2001 From: Alexandra Johnson Date: Tue, 9 Apr 2019 17:50:02 -0700 Subject: [PATCH 2/9] pkg/api/api.proto -> pkg/api/v1alpha1/api.proto --- README.md | 2 +- docs/developer-guide.md | 2 +- docs/rest-api.md | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 8bccc713914..a6ca77e61c9 100644 --- a/README.md +++ b/README.md @@ -61,7 +61,7 @@ A Suggestion is an algorithm to construct a parameter set. Currently Katib suppo ## Components in Katib Katib consists of several components as shown below. Each component is running on k8s as a deployment. -Each component communicates with others via GRPC and the API is defined at `pkg/api/api.proto`. +Each component communicates with others via GRPC and the API is defined at `pkg/api/v1alpha1/api.proto`. - vizier: main components. - vizier-core : API server of vizier. diff --git a/docs/developer-guide.md b/docs/developer-guide.md index fff36055665..ec8dd116c87 100644 --- a/docs/developer-guide.md +++ b/docs/developer-guide.md @@ -22,7 +22,7 @@ You can build all images from source. ## Implement new suggestion algorithm -Suggestion API is defined as GRPC service at `pkg/api/api.proto`. Source code is [here](https://github.com/kubeflow/katib/blob/master/pkg/api/api.proto). You can attach new algorithm easily. +Suggestion API is defined as GRPC service at `pkg/api/v1alpha1/api.proto`. Source code is [here](https://github.com/kubeflow/katib/blob/master/pkg/api/v1alpha1/api.proto). You can attach new algorithm easily. - implement suggestion API - make k8s service named vizier-suggestion-{ algorithm-name } and expose port 6789 diff --git a/docs/rest-api.md b/docs/rest-api.md index 591d7fb9257..0da9cc43063 100644 --- a/docs/rest-api.md +++ b/docs/rest-api.md @@ -1,6 +1,6 @@ -## REST API +## REST API -For each RPC service, we define an equivalent HTTP REST API method using [grpc-gateway](https://github.com/grpc-ecosystem/grpc-gateway). The mapping between service and REST API method can be found in this [file](https://github.com/kubeflow/katib/blob/master/pkg/api/api.proto). The mapping includes the URL path, query parameters and request body. You can read more details on the supported methods and binding options at this [link](https://cloud.google.com/service-infrastructure/docs/service-management/reference/rpc/google.api#http) +For each RPC service, we define an equivalent HTTP REST API method using [grpc-gateway](https://github.com/grpc-ecosystem/grpc-gateway). The mapping between service and REST API method can be found in this [file](https://github.com/kubeflow/katib/blob/master/pkg/api/v1alpha1/api.proto). The mapping includes the URL path, query parameters and request body. You can read more details on the supported methods and binding options at this [link](https://cloud.google.com/service-infrastructure/docs/service-management/reference/rpc/google.api#http) ## Examples From c19ea4e71e531c7d606ac576e94630a5ce619b09 Mon Sep 17 00:00:00 2001 From: Alexandra Johnson Date: Tue, 9 Apr 2019 17:51:08 -0700 Subject: [PATCH 3/9] pkg/api/gen-doc/api.md -> pkg/api/v1alpha1/gen-doc/api.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a6ca77e61c9..9f84ce01d94 100644 --- a/README.md +++ b/README.md @@ -90,7 +90,7 @@ You can visualize general trend of Hyper parameter space and each training histo ## API Documentation -Please refer to [api.md](./pkg/api/gen-doc/api.md). +Please refer to [api.md](./pkg/api/v1alpha1/gen-doc/api.md). ## Quickstart to run tfjob and pytorch operator jobs in Katib From 7ed905d7eff54e339b848a66f0fec987cc420548 Mon Sep 17 00:00:00 2001 From: Alexandra Johnson Date: Tue, 9 Apr 2019 18:01:24 -0700 Subject: [PATCH 4/9] Links in pkg/api/README.md need to be doubled up for alpha1 and alpha2 --- pkg/api/README.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/pkg/api/README.md b/pkg/api/README.md index dce7b5d8305..0213ab521f9 100644 --- a/pkg/api/README.md +++ b/pkg/api/README.md @@ -4,10 +4,17 @@ Manager service is an API for core component. Suggestion service is an API for Suggestion services. EarlyStopping service is an API for EarlyStopping services. -## Documentation -Please refer to [api.md](./gen-doc/api.md). +## v1alpha1 Documentation +Please refer to the `api.md`: + * [v1alpha1 documentation](./v1alpha1/gen-doc/api.md) + * [v1alpha2 documentation](./v1alpha2/gen-doc/api.md) ## Update API and generate documents -When you want to edit API, please only edit [api.proto](./api.proto) and generate other file from it. -Documents are also generated from [api.proto](./api.proto) by [protoc-gen-doc](https://github.com/pseudomuto/protoc-gen-doc). -By [build.sh](./build.sh), you can update every file from api.proto and generate docs. +When you want to edit the API, please only edit the corresponding `api.proto` and generate other files from it: + * [v1alpha1/api.proto](./v1alpha1/api.proto) + * [v1alpha2/api.proto](./v1alpha2/api.proto) + +Documents are also generated from `api.proto` by [protoc-gen-doc](https://github.com/pseudomuto/protoc-gen-doc). +Running `build.sh` can update every file from `api.proto` and generate docs: + * [v1alpha1/build.sh](./v1alpha1/build.sh) + * [v1alpha2/build.sh](./v1alpha2/build.sh) From 3bb70dbcd21154f771ad8267e35e093755efcb3a Mon Sep 17 00:00:00 2001 From: Alexandra Johnson Date: Tue, 9 Apr 2019 18:09:20 -0700 Subject: [PATCH 5/9] manifests -> manifests/v1alpha1 --- README.md | 4 ++-- examples/v1alpha1/README.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 9f84ce01d94..29f45193570 100644 --- a/README.md +++ b/README.md @@ -158,7 +158,7 @@ spec: Create this pv after deploying Katib package ``` -kubectl create -f https://raw.githubusercontent.com/kubeflow/katib/master/manifests/pv/pv.yaml +kubectl create -f https://raw.githubusercontent.com/kubeflow/katib/master/manifests/v1alpha1/pv/pv.yaml ``` ### Running examples @@ -359,7 +359,7 @@ ks delete ${KF_ENV} -c tf-job-operator If you create pv for Katib, delete it ``` -kubectl delete -f https://raw.githubusercontent.com/kubeflow/katib/master/manifests/pv/pv.yaml +kubectl delete -f https://raw.githubusercontent.com/kubeflow/katib/master/manifests/v1alpha1/pv/pv.yaml ``` If you deploy Ambassador, delete it diff --git a/examples/v1alpha1/README.md b/examples/v1alpha1/README.md index 403f2b0b3fc..5353786571a 100644 --- a/examples/v1alpha1/README.md +++ b/examples/v1alpha1/README.md @@ -369,7 +369,7 @@ batch2 accuracy=0.8 Validation-accuracy=0.75 ``` The metrics collector will collect all logs of metrics. -The manifest of metrics collector is also generated from template and defined [here](/manifests/studyjobcontroller/metricsControllerConfigMap.yaml). +The manifest of metrics collector is also generated from template and defined [here](./manifests/v1alpha1/studyjobcontroller/metricsControllerConfigMap.yaml). You can add your template and specify `spec.metricsCollectorSpec.metricsCollectorTemplatePath` in a studyjob manifest. ### TF Event File Metrics Collector From c21969cb06056f96a0691ebc26b0a1097411f3a9 Mon Sep 17 00:00:00 2001 From: Alexandra Johnson Date: Wed, 10 Apr 2019 11:24:25 -0700 Subject: [PATCH 6/9] Fix another examples link --- examples/v1alpha1/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/v1alpha1/README.md b/examples/v1alpha1/README.md index 5353786571a..1ffaf0739f4 100644 --- a/examples/v1alpha1/README.md +++ b/examples/v1alpha1/README.md @@ -199,7 +199,7 @@ The templates are defined in [ConfigMap](./workerConfigMap.yaml). There are two templates: defaultWorkerTemplate.yaml and gpuWorkerTemplate.yaml. You can add your template for worker. Then you should specify the template in your studyjob spec. -[This example](/examples/gpu-example.yaml) uses `gpuWorkerTemplate.yaml`. +[This example](/examples/v1alpha1/gpu-example.yaml) uses `gpuWorkerTemplate.yaml`. You can apply it same as other examples. ``` $ kubectl apply -f gpu-example.yaml From 8474e3c94cd65eb1b9d8ff64054f001df576dfa4 Mon Sep 17 00:00:00 2001 From: Alexandra Johnson Date: Wed, 10 Apr 2019 11:24:39 -0700 Subject: [PATCH 7/9] No more relative link --- examples/v1alpha1/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/v1alpha1/README.md b/examples/v1alpha1/README.md index 1ffaf0739f4..cd2f009fac3 100644 --- a/examples/v1alpha1/README.md +++ b/examples/v1alpha1/README.md @@ -369,7 +369,7 @@ batch2 accuracy=0.8 Validation-accuracy=0.75 ``` The metrics collector will collect all logs of metrics. -The manifest of metrics collector is also generated from template and defined [here](./manifests/v1alpha1/studyjobcontroller/metricsControllerConfigMap.yaml). +The manifest of metrics collector is also generated from template and defined [here](/manifests/v1alpha1/studyjobcontroller/metricsControllerConfigMap.yaml). You can add your template and specify `spec.metricsCollectorSpec.metricsCollectorTemplatePath` in a studyjob manifest. ### TF Event File Metrics Collector From e546268a2626f0fd8df9a68ef0c559a602e1011a Mon Sep 17 00:00:00 2001 From: Alexandra Johnson Date: Wed, 10 Apr 2019 11:25:28 -0700 Subject: [PATCH 8/9] rename header --- pkg/api/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pkg/api/README.md b/pkg/api/README.md index 0213ab521f9..6bafe5a4289 100644 --- a/pkg/api/README.md +++ b/pkg/api/README.md @@ -4,7 +4,7 @@ Manager service is an API for core component. Suggestion service is an API for Suggestion services. EarlyStopping service is an API for EarlyStopping services. -## v1alpha1 Documentation +## Documentation Please refer to the `api.md`: * [v1alpha1 documentation](./v1alpha1/gen-doc/api.md) * [v1alpha2 documentation](./v1alpha2/gen-doc/api.md) From 11045609b030992f13d8eb6c1d96ba5a5e6e512e Mon Sep 17 00:00:00 2001 From: Alexandra Johnson Date: Wed, 10 Apr 2019 11:27:06 -0700 Subject: [PATCH 9/9] update scripts link --- docs/developer-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer-guide.md b/docs/developer-guide.md index ec8dd116c87..d5f6d36d6b6 100644 --- a/docs/developer-guide.md +++ b/docs/developer-guide.md @@ -17,7 +17,7 @@ dep ensure --vendor-only You can build all images from source. ```bash -./scripts/build.sh +./scripts/v1alpha1/build.sh ``` ## Implement new suggestion algorithm