Fix operator updates on OLM

Summary:
Noticed that operators were actually struggling to update to the latest operator version...
After doing some digging, it turns out it was unhappy because the latest operator version has new labels and deployment labels are immutable.
luckily, we can still move forward and update from this case: operator-framework/operator-lifecycle-manager#952
OLM assumes the deployment name is the same, so it tries to do a rolling deploy. however, changing the deployment name indicates to OLM that we want to completely replace the deployment.

suggestions for a better new deployment name are appreciated.
also fixed a bug with the release build where the rc's prev versions were incorrect.

Test Plan:
created test plan in checklist for verifying operator updates, and ran through the test plan:
https://www.notion.so/pixielabs/Operator-Release-Checklist-a705283f190c4c0aa127f9439bb34180

Reviewers: vihang, zasgar

Reviewed By: vihang

Differential Revision: https://phab.corp.pixielabs.ai/D9533

GitOrigin-RevId: 561964e

ci/operator_build_release.sh

@@ -27,17 +27,28 @@ echo "The release tag is: ${release_tag}"
 bazel run -c opt //src/utils/artifacts/versions_gen:versions_gen -- \
       --repo_path "${repo_path}" --artifact_name operator --versions_file "${versions_file}"
 
+# Find the previous bundle version, which this release should replace.
+tags=$(git for-each-ref --sort='-*authordate' --format '%(refname:short)' refs/tags \
+    | grep "release/operator" | grep -v "\-")
+
 public="True"
 image_path="gcr.io/pixie-oss/pixie-prod/operator/operator_image:${release_tag}"
 channel="stable"
 channels="stable,dev"
 bucket="pixie-dev-public"
+# The previous version should be the 2nd item in the tags. Since this is a release build,
+# the first item in the tag is the current release.
+prev_tag=$(echo "$tags" | sed -n '2 p')
+
 if [[ $release_tag == *"-"* ]]; then
   public="False"
   image_path="gcr.io/pixie-oss/pixie-dev/operator/operator_image:${release_tag}"
   channel="dev"
   channels="dev"
   bucket="pixie-prod-artifacts"
+# The previous version should be the 1st item in the tags. Since this is a non-release build,
+# the first item in the tags is the previous release.
+  prev_tag=$(echo "$tags" | sed -n '1 p')
 fi
 
 # Push operator image.
@@ -55,10 +66,6 @@ bundle_version=$(echo "${release_tag}" | tr '[:upper:]' '[:lower:]')
 # about how the operator should be deployed.
 mkdir "${tmp_dir}/manifests"
 
-# Find the previous bundle version, which this release should replace.
-tags=$(git for-each-ref --sort='-*authordate' --format '%(refname:short)' refs/tags \
-    | grep "release/operator" | grep -v "\-")
-prev_tag=$(echo "$tags" | sed -n '2 p')
 previous_version=${prev_tag//*\/v/}
 
 kustomize build "$(pwd)/k8s/operator/crd/base" > "${kustomize_dir}/crd.yaml"