-
Notifications
You must be signed in to change notification settings - Fork 104
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make baremetal targets more resilient to re-running #1004
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -126,7 +126,8 @@ BMAAS_ROUTE_LIBVIRT_NETWORKS ?= ${BMAAS_NETWORK_NAME},crc,default | |
DATAPLANE_PLAYBOOK ?= osp.edpm.download_cache | ||
DATAPLANE_CUSTOM_SERVICE_RUNNER_IMG ?=quay.io/openstack-k8s-operators/openstack-ansibleee-runner:latest | ||
BM_NETWORK_NAME ?=default | ||
BM_INSTANCE_NAME_PREFIX ?=edpm-compute | ||
BM_INSTANCE_NAME_PREFIX ?=edpm-compute-baremetal | ||
BM_INSTANCE_NAME_SUFFIX ?=0 | ||
BM_NODE_COUNT ?=1 | ||
BM_ROOT_PASSWORD_SECRET ?= | ||
BMH_NAMESPACE ?=openstack | ||
|
@@ -377,25 +378,39 @@ edpm_baremetal_compute: export OPERATOR_NAME=openstack | |
edpm_baremetal_compute: export NAMESPACE=${BMH_NAMESPACE} | ||
edpm_baremetal_compute: export BMAAS_NETWORK_NAME=${BM_NETWORK_NAME} | ||
edpm_baremetal_compute: export BMAAS_INSTANCE_NAME_PREFIX=${BM_INSTANCE_NAME_PREFIX} | ||
edpm_baremetal_compute: export BMAAS_INSTANCE_NAME_SUFFIX=${BM_INSTANCE_NAME_SUFFIX} | ||
edpm_baremetal_compute: export BMAAS_NODE_COUNT=${BM_NODE_COUNT} | ||
edpm_baremetal_compute: export NODE_COUNT=${BM_NODE_COUNT} | ||
edpm_baremetal_compute: export DEPLOY_DIR=../out/bmaas | ||
edpm_baremetal_compute: ## Create virtual baremetal for the dataplane | ||
$(eval $(call vars)) | ||
make bmaas_virtual_bms | ||
make bmaas_sushy_emulator_wait | ||
bash scripts/edpm-compute-baremetal.sh --create | ||
if [ "${NODE_COUNT}" = "1" ]; then \ | ||
bash scripts/edpm-compute-baremetal.sh --create ; \ | ||
else \ | ||
for INDEX in $(shell seq 0 $$((${NODE_COUNT} -1))) ; do \ | ||
export BMAAS_INSTANCE_NAME_SUFFIX=$$INDEX ; \ | ||
bash scripts/edpm-compute-baremetal.sh --create ; \ | ||
done \ | ||
fi | ||
|
||
edpm_baremetal_compute_cleanup: export BMAAS_NETWORK_NAME=${BM_NETWORK_NAME} | ||
edpm_baremetal_compute_cleanup: export NAMESPACE=${BMH_NAMESPACE} | ||
edpm_baremetal_compute_cleanup: export BMAAS_INSTANCE_NAME_PREFIX=${BM_INSTANCE_NAME_PREFIX} | ||
edpm_baremetal_compute_cleanup: export BMAAS_INSTANCE_NAME_SUFFIX=${BM_INSTANCE_NAME_SUFFIX} | ||
edpm_baremetal_compute_cleanup: export BMAAS_NODE_COUNT=${BM_NODE_COUNT} | ||
edpm_baremetal_compute_cleanup: export NODE_COUNT=${BM_NODE_COUNT} | ||
edpm_baremetal_compute_cleanup: ## Cleanup dataplane with BMAAS | ||
$(eval $(call vars)) | ||
scripts/edpm-compute-baremetal.sh --cleanup | ||
pushd .. && rm -Rf out/edpm || true && popd | ||
make bmaas_sushy_emulator_cleanup || true | ||
if [ "${NODE_COUNT}" = "1" ]; then \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As we're changing the name of the vms and bmhs, cleanup won't work with things created with earlier version of the script. They have to be manually deleted. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think that would still be possible with using the env vars to set the names. However, I don't think we need to maintain backwards compatibility here anyway. |
||
bash scripts/edpm-compute-baremetal.sh --cleanup ; \ | ||
else \ | ||
for INDEX in $(shell seq 0 $$((${NODE_COUNT} -1))) ; do \ | ||
export BMAAS_INSTANCE_NAME_SUFFIX=$$INDEX ; \ | ||
bash scripts/edpm-compute-baremetal.sh --cleanup ; \ | ||
done \ | ||
fi | ||
make bmaas_virtual_bms_cleanup || true | ||
|
||
.PHONY: edpm_compute | ||
|
@@ -662,27 +677,42 @@ bmaas_metallb_cleanup: | |
bmaas_virtual_bms: export NODE_COUNT = ${BMAAS_NODE_COUNT} | ||
bmaas_virtual_bms: export NETWORK_NAME = ${BMAAS_NETWORK_NAME} | ||
bmaas_virtual_bms: export NODE_NAME_PREFIX = ${BMAAS_INSTANCE_NAME_PREFIX} | ||
bmaas_virtual_bms: export NODE_NAME_SUFFIX = ${BMAAS_INSTANCE_NAME_SUFFIX} | ||
bmaas_virtual_bms: export MEMORY = ${BMAAS_INSTANCE_MEMORY} | ||
bmaas_virtual_bms: export VCPUS = ${BMAAS_INSTANCE_VCPUS} | ||
bmaas_virtual_bms: export DISK_SIZE = ${BMAAS_INSTANCE_DISK_SIZE} | ||
bmaas_virtual_bms: export OS_VARIANT = ${BMAAS_INSTANCE_OS_VARIANT} | ||
bmaas_virtual_bms: export VIRT_TYPE = ${BMAAS_INSTANCE_VIRT_TYPE} | ||
bmaas_virtual_bms: export NET_MODEL = ${BMAAS_INSTANCE_NET_MODEL} | ||
bmaas_virtual_bms: ## Create libvirt VM for BMaaS | ||
scripts/bmaas/vbm-setup.sh --create | ||
if [ ${NODE_COUNT} -eq 1 ]; then \ | ||
scripts/bmaas/vbm-setup.sh --create ; \ | ||
else \ | ||
for INDEX in $(shell seq 0 $$((${NODE_COUNT} -1))) ; do \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Calling these scripts as many times as the number of nodes is possibly little inefficient, though does not matter probably as no one would be deploying large number of nodes with install_yamls. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why do we need to iterate in the Makefile though? The
Then we can just handle the Just avoids re-running the script like Rabi is mentioning/ There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess the whole check isn't really needed in that scenario actually, because There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This PR changes vbm-setup.sh to drop --num-nodes, so it manages just a single VM. That change is already included in this PR. Are you asking me to drop that change from the PR? The only inefficient part of calling the script twice would be setting up the libvirt logging, not really something I'd optimize in a dev repo, but we can go back to the way it was if that is wanted. gen-edpm-node.sh manages 1 vm at a time. That is used for networker and bootc vms already, with the Makefile controlling the looping. So, I was changing the baremetal scripts to work in a similar way. gen-edpm-node-bgp.sh is hardcoded to manage 3 at a time, so that's a different way as well. Whichever way it's done, I'd like to see these things consistent. |
||
set -x ; \ | ||
NODE_NAME_SUFFIX=$$INDEX scripts/bmaas/vbm-setup.sh --create ; \ | ||
done \ | ||
fi | ||
|
||
.PHONY: bmaas_virtual_bms_cleanup | ||
bmaas_virtual_bms_cleanup: export NODE_COUNT = ${BMAAS_NODE_COUNT} | ||
bmaas_virtual_bms_cleanup: export NETWORK_NAME = ${BMAAS_NETWORK_NAME} | ||
bmaas_virtual_bms_cleanup: export NODE_NAME_PREFIX = ${BMAAS_INSTANCE_NAME_PREFIX} | ||
bmaas_virtual_bms_cleanup: export NODE_NAME_SUFFIX = ${BMAAS_INSTANCE_NAME_SUFFIX} | ||
bmaas_virtual_bms_cleanup: export MEMORY = ${BMAAS_INSTANCE_MEMORY} | ||
bmaas_virtual_bms_cleanup: export VCPUS = ${BMAAS_INSTANCE_VCPUS} | ||
bmaas_virtual_bms_cleanup: export DISK_SIZE = ${BMAAS_INSTANCE_DISK_SIZE} | ||
bmaas_virtual_bms_cleanup: export OS_VARIANT = ${BMAAS_INSTANCE_OS_VARIANT} | ||
bmaas_virtual_bms_cleanup: export VIRT_TYPE = ${BMAAS_INSTANCE_VIRT_TYPE} | ||
bmaas_virtual_bms_cleanup: export NET_MODEL = ${BMAAS_INSTANCE_NET_MODEL} | ||
bmaas_virtual_bms_cleanup: ## Cleanup libvirt VM for BMaaS | ||
scripts/bmaas/vbm-setup.sh --cleanup | ||
if [ ${NODE_COUNT} -eq 1 ]; then \ | ||
scripts/bmaas/vbm-setup.sh --cleanup ; \ | ||
else \ | ||
for INDEX in $(shell seq 0 $$((${NODE_COUNT} -1))) ; do \ | ||
NODE_NAME_SUFFIX=$$INDEX scripts/bmaas/vbm-setup.sh --cleanup ; \ | ||
done \ | ||
fi | ||
|
||
.PHONY: bmaas_sushy_emulator | ||
bmaas_sushy_emulator: export NODE_NAME_PREFIX = ${BMAAS_INSTANCE_NAME_PREFIX} | ||
|
@@ -697,17 +727,20 @@ bmaas_sushy_emulator: export SUSHY_EMULATOR_OS_CLOUD = ${BMAAS_SUSHY_EMULATOR_OS | |
bmaas_sushy_emulator: export SUSHY_EMULATOR_OS_CLIENT_CONFIG_FILE = ${BMAAS_SUSHY_EMULATOR_OS_CLIENT_CONFIG_FILE} | ||
bmaas_sushy_emulator: export REDFISH_USERNAME = ${BMAAS_REDFISH_USERNAME} | ||
bmaas_sushy_emulator: export REDFISH_PASSWORD = ${BMAAS_REDFISH_PASSWORD} | ||
bmaas_sushy_emulator: export DEPLOY_DIR=../out/bmaas | ||
bmaas_sushy_emulator: ## Create BMaaS sushy-emulator (Virtual RedFish) | ||
scripts/bmaas/sushy-emulator.sh --create | ||
|
||
.PHONY: bmaas_sushy_emulator_cleanup | ||
bmaas_sushy_emulator_cleanup: export NODE_NAME_PREFIX = ${BMAAS_INSTANCE_NAME_PREFIX} | ||
bmaas_sushy_emulator_cleanup: export NODE_NAME_SUFFIX = ${BMAAS_INSTANCE_NAME_SUFFIX} | ||
bmaas_sushy_emulator_cleanup: export LIBVIRT_USER = ${BMAAS_LIBVIRT_USER} | ||
bmaas_sushy_emulator_cleanup: export SUSHY_EMULATOR_NAMESPACE = ${BMAAS_SUSHY_EMULATOR_NAMESPACE} | ||
bmaas_sushy_emulator_cleanup: export SUSHY_EMULATOR_DRIVER = ${BMAAS_SUSHY_EMULATOR_DRIVER} | ||
bmaas_sushy_emulator_cleanup: export SUSHY_EMULATOR_OS_CLOUD = ${BMAAS_SUSHY_EMULATOR_OS_CLOUD} | ||
bmaas_sushy_emulator_cleanup: export REDFISH_USERNAME = ${BMAAS_REDFISH_USERNAME} | ||
bmaas_sushy_emulator_cleanup: export REDFISH_PASSWORD = ${BMAAS_REDFISH_PASSWORD} | ||
bmaas_sushy_emulator_cleanup: export DEPLOY_DIR=../out/bmaas | ||
bmaas_sushy_emulator_cleanup: ## Cleanup BMaaS sushy-emulator (Virtual RedFish) | ||
scripts/bmaas/sushy-emulator.sh --cleanup | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Though you can now re-run to create additional BMHs, as we cleanup the existing OpenstackDataPlaneNodeSet/OpenStackDataPlaneDeployments with
edpm_deploy_cleanup
[1] that would deprovision the all existing BMHs as well, before provisioning them again.[1] https://github.com/openstack-k8s-operators/install_yamls/blob/main/Makefile#L922
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
true. I would actually drop the cleanup deps from edpm_baremetal_deploy and edpm_deploy. I usually use the existing resources already written out, customize them as needed, and use them with oc directly. I guess it depends how far we want to take it in terms of making the targets work without cleanup or making them re-runnable.