Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow BBR post unlock to proceed if CC api is remotely available #311

Merged
merged 1 commit into from
Apr 25, 2023
Merged

Allow BBR post unlock to proceed if CC api is remotely available #311

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions jobs/cloud_controller_ng/templates/post-backup-unlock.sh.erb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ source /var/vcap/packages/capi_utils/syslog_utils.sh
<% (1..(p("cc.jobs.local.number_of_workers"))).each do |index| %>
monit_start_job cloud_controller_worker_local_<%= index %>
<% end %>
wait_for_server_to_become_healthy <%= "localhost:#{p("cc.external_port")}/healthz" %> <%= p("cc.post_bbr_healthcheck_timeout_in_seconds") %>
sleep 30
set +e
Copy link
Member

@sethboyles sethboyles Apr 24, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this script is even set to -e because #L13 might return 0 after setting +e

wait_for_server_to_become_healthy_without_setminuse <%= "localhost:#{p("cc.external_port")}/healthz" %> <%= p("cc.post_bbr_healthcheck_timeout_in_seconds") %>
set -e
if [ $? -eq 0 ]; then
wait_for_server_to_become_healthy <%= "#{p("cc.external_protocol")}://#{p("cc.external_host")}.#{p("system_domain")}/info" %> <%= p("cc.post_bbr_healthcheck_timeout_in_seconds") %>
fi
<% end %>
14 changes: 14 additions & 0 deletions src/capi_utils/monit_utils.sh
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side note, but the way this function can return 0 before resetting minus e is suspect: https://github.com/cloudfoundry/capi-release/pull/311/files#diff-c0db106c51e4e0b0add9487bf8b3753cbd75ac3a11d5a0c3fdfeaadc46598a2cR45

Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,20 @@ function wait_for_server_to_become_healthy() {
return 1
}

function wait_for_server_to_become_healthy_without_setminuse() {
local url=$1
local timeout=$2
for _ in $(seq "${timeout}"); do
curl -k -f --connect-timeout 1 "${url}" > /dev/null 2>&1
if [ $? -eq 0 ]; then
return 0
fi
sleep 1
done

echo "Endpoint ${url} failed to become healthy after ${timeout} seconds"
return 1
}
# monit_monitor_job
#
# @param job_name
Expand Down