Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

Add an operator [ paictl conf update ] to upload configuration to k8s #1431

Merged
merged 46 commits into from
Sep 29, 2018

Conversation

ydye
Copy link
Contributor

@ydye ydye commented Sep 25, 2018

Progress

  • paictl config update -p /path/to/local/configuration [-c kubeconfig-path ]
  • paictl config update -e external-storage-config [-c kubeconfig-path]
  • paictl config update [-c kubeconfig-path]

image

  • paiclt.py config external-config-update -e external-configuration [-c kube-config]

image

  • paictl.py config get -o /path/to/output [-c kube-config]

Change

  1. With the provided external-configuration-storage conf, you could get your cluster-configuration from the service like git, and then upload it to k8s cluster.
  2. The cluster-configuration will be updated into a configmap. And all service in the k8s cluster could get it from configmap volume.
  3. Tool paictl will get the configuration from k8s-configmap. So next time, admin should update their cluster configuration from external storage to the cluster. Then manage the openpai cluster.

@coveralls
Copy link

coveralls commented Sep 25, 2018

Coverage Status

Coverage increased (+0.07%) to 51.546% when pulling 5a1b7fd on yuye/external-git into b5a674a on master.

@@ -64,6 +64,9 @@ sudo chown core:core -R /pathHadoop/
QUICK_START_PATH=${JENKINS_HOME}/${BED}/singlebox/quick-start
CONFIG_PATH=${JENKINS_HOME}/${BED}/singlebox/cluster-configuration

# Install necessary python package
sudo pip install GitPython
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we remove the line? Noticed that you have updated the dependencies in dev-box.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No we can't. paictl need import it. You will call paictl to generate cluster-configuration out of the dev-box.

@ydye ydye requested a review from xudifsd September 26, 2018 08:58
@ydye
Copy link
Contributor Author

ydye commented Sep 27, 2018

#1440

Copy link

@sterowang sterowang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:




def open(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe you can use __enter__ and __exit__, which is more elegant, and can clean storage in exit.

Same applies to local_storage.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed


self.time = str(int(time.time()))
self.logger = logging.getLogger(__name__)
self.local_store = "{0}-{1}".format(local_store, self.time)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not using tempfile

Copy link
Contributor Author

@ydye ydye Sep 28, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will have to remove the temp directory created by tempfile.mkdtemp([suffix=''[, prefix='tmp'[, dir=None]]]) . And I think it's same as what we have done. Here I prefer manually handle it.


def git_clone(self):
try:
self.repo = git.Repo.clone_from(self.repo_url, self.local_store, branch=self.branch)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we just clone with --depth=1, since we requires no history.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed.

@ydye ydye merged commit a47ed4a into master Sep 29, 2018
@ydye ydye deleted the yuye/external-git branch September 29, 2018 02:10
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants