fix: Fix GitHub Actions to pass authentication (#4963)

* fix: Update workflows to also use workflow_call Signed-off-by: Francisco Javier Arceo <[email protected]> * fix: Also adding workflow_call to release version Signed-off-by: Francisco Javier Arceo <[email protected]> * fix: Adding type to release version input Signed-off-by: Francisco Javier Arceo <[email protected]> * chore: Fix operator release workflow (v0.43-branch) (#4937) fix operator release workflow Signed-off-by: Tommy Hughes <[email protected]> * fix: Updated type in yaml Signed-off-by: Francisco Javier Arceo <[email protected]> * fix: Updating publish.yml Signed-off-by: Francisco Javier Arceo <[email protected]> * fix: Update type in publish_images Signed-off-by: Francisco Javier Arceo <[email protected]> * fix: Updating publish helm charts Signed-off-by: Francisco Javier Arceo <[email protected]> * fix: Updating publish java sdk Signed-off-by: Francisco Javier Arceo <[email protected]> * fix: Updating environment variable in the build_wheels Signed-off-by: Francisco Javier Arceo <[email protected]> * updating input parameters to build wheels Signed-off-by: Francisco Javier Arceo <[email protected]> * formatting issue Signed-off-by: Francisco Javier Arceo <[email protected]> * chore: Fix operator release for 0.43.0 (v0.43-branch) (#4939) fix operator release for 0.43.0 Signed-off-by: Tommy Hughes <[email protected]> * renaming args and adding more Signed-off-by: Francisco Javier Arceo <[email protected]> * fix: Adding outputs to build_wheels workflow Signed-off-by: Francisco Javier Arceo <[email protected]> * reverting changes Signed-off-by: Francisco Javier Arceo <[email protected]> * chore: Add ability to change default feature-server image during Operator deploy (v0.43-branch) (#4941) chore: Add ability to change default feature-server image during Operator deploy (#4940) Signed-off-by: Tommy Hughes <[email protected]> * chore: CR image no longer needed for e2e (v0.43-branch) (#4943) CR image no longer needed for e2e Signed-off-by: Tommy Hughes <[email protected]> * chore: Fix publish, versions issue (#4944) fix publish, versions issue Signed-off-by: Tommy Hughes <[email protected]> * chore: Fix publish, versions issue (#4945) fix publish, versions issue Signed-off-by: Tommy Hughes <[email protected]> * chore: Fix publish, versions issue (#4946) fix publish, versions issue Signed-off-by: Tommy Hughes <[email protected]> * chore: Fix publish, versions issue (#4947) fix publish, versions issue Signed-off-by: Tommy Hughes <[email protected]> * fix publish, versions issue (#4948) Signed-off-by: Tommy Hughes <[email protected]> * fix: adding checkout to publish sdk Signed-off-by: Francisco Javier Arceo <[email protected]> * chore: Change get version workflow to an action (#4951) change get version workflow to an action Signed-off-by: Tommy Hughes <[email protected]> * fix: Release Fix - Added the workflow_call element in missing workflows (#4954) Added the workflow_call element in all the workflows wherever there is a workflow_dispatch element to see if that fixes the problem to be able to call the workflow from another workflow. Signed-off-by: lrangine <[email protected]> * chore: Build wheels fix (#4957) dev Signed-off-by: Tommy Hughes <[email protected]> * chore: Fix python-sdk workflow (#4958) dev Signed-off-by: Tommy Hughes <[email protected]> * fix: adding type and workflow_call to build_wheels.yml Signed-off-by: Francisco Javier Arceo <[email protected]> * chore: Remove checkout from get-version action (#4959) dev Signed-off-by: Tommy Hughes <[email protected]> * chore: Fixing build wheels pre-release image build (#4960) dev Signed-off-by: Tommy Hughes <[email protected]> * fix: Updating pypi upload to use legacy repo url (#4962) Signed-off-by: Francisco Javier Arceo <[email protected]> * fix: Updating repostiry to repostory url Signed-off-by: Francisco Javier Arceo <[email protected]> * converting to dash to see if that works Signed-off-by: Francisco Javier Arceo <[email protected]> * fix: Updating pypi publish action and adding name Signed-off-by: Francisco Javier Arceo <[email protected]> * fix: moving back to kebab case Signed-off-by: Francisco Javier Arceo <[email protected]> * fix: Trying to hardcode gh-action version Signed-off-by: Francisco Javier Arceo <[email protected]> * trying new version with oidc false Signed-off-by: Francisco Javier Arceo <[email protected]> * trying without attestations Signed-off-by: Francisco Javier Arceo <[email protected]> * reverting again to old action and removing checkout action Signed-off-by: Francisco Javier Arceo <[email protected]> * reverting code to existing master without actions checkout Signed-off-by: Francisco Javier Arceo <[email protected]> * logging token length to validate it and adding quotes to password Signed-off-by: Francisco Javier Arceo <[email protected]> * adding logging to the build wheels to see if secret is present there Signed-off-by: Francisco Javier Arceo <[email protected]> * adding secrets to jobs keyword Signed-off-by: Francisco Javier Arceo <[email protected]> * adding input parameters to build wheels and publish python sdk Signed-off-by: Francisco Javier Arceo <[email protected]> * addding inherit Signed-off-by: Francisco Javier Arceo <[email protected]> * commenting out the publish python sdk and adding inherit Signed-off-by: Francisco Javier Arceo <[email protected]> * commenting out the needs Signed-off-by: Francisco Javier Arceo <[email protected]> * removing commented out code and logging to incorporate code that worked Signed-off-by: Francisco Javier Arceo <[email protected]> * adding one more change to remove logging Signed-off-by: Francisco Javier Arceo <[email protected]> * removing changes outside of actions Signed-off-by: Francisco Javier Arceo <[email protected]> * removing makefile change Signed-off-by: Francisco Javier Arceo <[email protected]> * removing yamls Signed-off-by: Francisco Javier Arceo <[email protected]> * removing changes files and reverting move of file Signed-off-by: Francisco Javier Arceo <[email protected]> * reverting changed files Signed-off-by: Francisco Javier Arceo <[email protected]> --------- Signed-off-by: Francisco Javier Arceo <[email protected]> Signed-off-by: Tommy Hughes <[email protected]> Signed-off-by: lrangine <[email protected]> Co-authored-by: Tommy Hughes IV <[email protected]> Co-authored-by: lokeshrangineni <[email protected]>
feast-dev · Jan 23, 2025 · 22b9138 · 22b9138
1 parent b026d0c
commit 22b9138
Show file tree

Hide file tree

Showing 9 changed files with 264 additions and 159 deletions.
diff --git a/.github/actions/get-semantic-release-version/action.yml b/.github/actions/get-semantic-release-version/action.yml
@@ -0,0 +1,87 @@
+name: Get semantic release version
+description: ""
+inputs:
+  custom_version: # Optional input for a custom version
+    description: "Custom version to publish (e.g., v1.2.3) -- only edit if you know what you are doing"
+    required: false
+  token:
+    description: "Personal Access Token"
+    required: true
+    default: ""
+outputs:
+  release_version:
+    description: "The release version to use (e.g., v1.2.3)"
+    value: ${{ steps.get_release_version.outputs.release_version }}
+  version_without_prefix:
+    description: "The release version to use without 'v' (e.g., 1.2.3)"
+    value: ${{ steps.get_release_version_without_prefix.outputs.version_without_prefix }}
+  highest_semver_tag:
+    description: "The highest semantic version tag without the 'v' prefix (e.g., 1.2.3)"
+    value: ${{ steps.get_highest_semver.outputs.highest_semver_tag }}
+runs:
+  using: composite
+  steps:
+    - name: Get release version
+      id: get_release_version
+      shell: bash
+      env:
+        GITHUB_TOKEN: ${{ inputs.token }}
+        GIT_AUTHOR_NAME: feast-ci-bot
+        GIT_AUTHOR_EMAIL: [email protected]
+        GIT_COMMITTER_NAME: feast-ci-bot
+        GIT_COMMITTER_EMAIL: [email protected]
+      run: |
+        if [[ -n "${{ inputs.custom_version }}" ]]; then
+          VERSION_REGEX="^v[0-9]+\.[0-9]+\.[0-9]+$"
+          echo "Using custom version: ${{ inputs.custom_version }}"
+          if [[ ! "${{ inputs.custom_version }}" =~ $VERSION_REGEX ]]; then
+            echo "Error: custom_version must match semantic versioning (e.g., v1.2.3)."
+            exit 1
+          fi
+          echo "::set-output name=release_version::${{ inputs.custom_version }}"
+        elif [[ "${GITHUB_REF}" == refs/tags/* ]]; then
+          echo "Using tag reference: ${GITHUB_REF#refs/tags/}"
+          echo "::set-output name=release_version::${GITHUB_REF#refs/tags/}"
+        else
+          echo "Defaulting to branch name: ${GITHUB_REF#refs/heads/}"
+          echo "::set-output name=release_version::${GITHUB_REF#refs/heads/}"
+        fi
+    - name: Get release version without prefix
+      id: get_release_version_without_prefix
+      shell: bash
+      env:
+        RELEASE_VERSION: ${{ steps.get_release_version.outputs.release_version }}
+      run: |
+        if [[ "${RELEASE_VERSION}" == v* ]]; then
+          echo "::set-output name=version_without_prefix::${RELEASE_VERSION:1}"
+        else
+          echo "::set-output name=version_without_prefix::${RELEASE_VERSION}"
+        fi
+    - name: Get highest semver
+      id: get_highest_semver
+      shell: bash
+      env:
+        RELEASE_VERSION: ${{ steps.get_release_version.outputs.release_version }}
+      run: |
+        if [[ -n "${{ inputs.custom_version }}" ]]; then
+          HIGHEST_SEMVER_TAG="${{ inputs.custom_version }}"
+          echo "::set-output name=highest_semver_tag::$HIGHEST_SEMVER_TAG"
+          echo "Using custom version as highest semantic version: $HIGHEST_SEMVER_TAG"
+        else 
+          source infra/scripts/setup-common-functions.sh
+          SEMVER_REGEX='^v[0-9]+\.[0-9]+\.[0-9]+(-([0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*))?$'
+          if echo "${RELEASE_VERSION}" | grep -P "$SEMVER_REGEX" &>/dev/null ; then
+            echo ::set-output name=highest_semver_tag::$(get_tag_release -m)
+            echo "Using infra/scripts/setup-common-functions.sh to generate highest semantic version: $HIGHEST_SEMVER_TAG"
+          fi
+        fi
+    - name: Check output
+      shell: bash
+      env:
+        RELEASE_VERSION: ${{ steps.get_release_version.outputs.release_version }}
+        VERSION_WITHOUT_PREFIX: ${{ steps.get_release_version_without_prefix.outputs.version_without_prefix }}
+        HIGHEST_SEMVER_TAG: ${{ steps.get_highest_semver.outputs.highest_semver_tag }}
+      run: |
+        echo $RELEASE_VERSION
+        echo $VERSION_WITHOUT_PREFIX
+        echo $HIGHEST_SEMVER_TAG
diff --git a/.github/workflows/build_wheels.yml b/.github/workflows/build_wheels.yml
@@ -1,30 +1,33 @@
-name: build_wheels
+name: build wheels
 
 # Call this workflow from other workflows in the repository by specifying "uses: ./.github/workflows/build_wheels.yml"
 # Developers who are starting a new release should use this workflow to ensure wheels will be built correctly.
 # Devs should check out their fork, add a tag to the last master commit on their fork, and run the release off of their fork on the added tag to ensure wheels will be built correctly.
 on:
-  workflow_dispatch:
-    tags:
-      - 'v*.*.*'
-  workflow_call:
+  workflow_dispatch: # Allows manual trigger of the workflow
     inputs:
-      release_version:
-        description: 'The release version to use (e.g., v1.2.3)'
+      custom_version: # Optional input for a custom version
+        description: 'Custom version to publish (e.g., v1.2.3) -- only edit if you know what you are doing'
+        required: false
+        type: string
+      token:
+        description: 'Personal Access Token'
         required: true
+        default: ""
+        type: string
+  workflow_call:
+    inputs:
+      custom_version: # Optional input for a custom version
+        description: 'Custom version to publish (e.g., v1.2.3) -- only edit if you know what you are doing'
+        required: false
         type: string
-      highest_semver_tag:
-        description: 'The highest semantic version tag without the "v" prefix (e.g., 1.2.3)'
+      token:
+        description: 'Personal Access Token'
         required: true
+        default: ""
         type: string
 
 jobs:
-  get-version:
-    uses: ./.github/workflows/get_semantic_release_version.yml
-    with:
-      custom_version: ${{ github.event.inputs.custom_version }}
-      token: ${{ github.event.inputs.token }}
-
   build-python-wheel:
     name: Build wheels
     runs-on: ubuntu-latest
@@ -42,10 +45,18 @@ jobs:
           registry-url: 'https://registry.npmjs.org'
       - name: Build UI
         run: make build-ui
+      - id: get-version
+        uses: ./.github/actions/get-semantic-release-version
+        with:
+          custom_version: ${{ github.event.inputs.custom_version }}
+          token: ${{ github.event.inputs.token }}
       - name: Build wheels
+        env:
+          VERSION: ${{ steps.get-version.outputs.release_version }}
+          PYPI_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
         run: |
           git fetch --tags
-          git checkout v0.42.0
+          git checkout ${VERSION}
           python -m pip install build
           python -m build --wheel --outdir wheelhouse/
       - uses: actions/upload-artifact@v4
@@ -69,11 +80,18 @@ jobs:
         with:
           node-version-file: './ui/.nvmrc'
           registry-url: 'https://registry.npmjs.org'
+      - id: get-version
+        uses: ./.github/actions/get-semantic-release-version
+        with:
+          custom_version: ${{ github.event.inputs.custom_version }}
+          token: ${{ github.event.inputs.token }}
       - name: Build and install dependencies
+        env:
+          VERSION: ${{ steps.get-version.outputs.release_version }}
         # There's a `git restore` in here because `make install-go-ci-dependencies` is actually messing up go.mod & go.sum.
         run: |
           git fetch --tags
-          git checkout v0.42.0
+          git checkout ${VERSION}
           pip install -U pip setuptools wheel twine
           make build-ui
           git status
@@ -89,11 +107,12 @@ jobs:
 
   # We add this step so the docker images can be built as part of the pre-release verification steps.
   build-docker-images:
+    name: Build Docker images
     runs-on: ubuntu-latest
-    needs: get-version
+    needs: [ build-python-wheel, build-source-distribution ]
     strategy:
       matrix:
-        component: [ feature-server, feature-server-java, feature-transformation-server, feast-operator ]
+        component: [ feature-server-dev, feature-server-java, feature-transformation-server, feast-operator ]
     env:
       REGISTRY: feastdev
     steps:
@@ -102,15 +121,23 @@ jobs:
         uses: docker/setup-qemu-action@v1
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v1
+      - id: get-version
+        uses: ./.github/actions/get-semantic-release-version
+        with:
+          custom_version: ${{ github.event.inputs.custom_version }}
+          token: ${{ github.event.inputs.token }}
       - name: Build image
+        env:
+          VERSION_WITHOUT_PREFIX: ${{ steps.get-version.outputs.version_without_prefix }}
+          RELEASE_VERSION: ${{ steps.get-version.outputs.release_version }}
         run: |
+          echo "Building docker image for ${{ matrix.component }} with version $VERSION_WITHOUT_PREFIX and release version $RELEASE_VERSION"
           make build-${{ matrix.component }}-docker REGISTRY=${REGISTRY} VERSION=${VERSION_WITHOUT_PREFIX}
-        env:
-          VERSION_WITHOUT_PREFIX: ${{ needs.get-version.outputs.version_without_prefix }}
 
   verify-python-wheels:
+    name: Verify Python wheels
     runs-on: ${{ matrix.os }}
-    needs: [ build-python-wheel, build-source-distribution, get-version ]
+    needs: [ build-python-wheel, build-source-distribution ]
     strategy:
       matrix:
         os: [ ubuntu-latest,  macos-13 ]
@@ -129,8 +156,8 @@ jobs:
         else
            echo "Succeeded!"
         fi
-      VERSION_WITHOUT_PREFIX: ${{ needs.get-version.outputs.version_without_prefix }}
     steps:
+      - uses: actions/checkout@v4
       - name: Setup Python
         id: setup-python
         uses: actions/setup-python@v5
@@ -160,7 +187,14 @@ jobs:
         if: ${{ matrix.from-source }}
         run: pip install dist/*tar.gz
       # Validate that the feast version installed is not development and is the correct version of the tag we ran it off of.
+      - id: get-version
+        uses: ./.github/actions/get-semantic-release-version
+        with:
+          custom_version: ${{ github.event.inputs.custom_version }}
+          token: ${{ github.event.inputs.token }}
       - name: Validate Feast Version
+        env:
+          VERSION_WITHOUT_PREFIX: ${{ steps.get-version.outputs.version_without_prefix }}
         run: |
           feast version
           if ! VERSION_OUTPUT=$(feast version); then

diff --git a/.github/workflows/get_semantic_release_version.yml b/.github/workflows/get_semantic_release_version.yml
diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml
@@ -9,42 +9,52 @@ on:
       custom_version: # Optional input for a custom version
         description: 'Custom version to publish (e.g., v1.2.3) -- only edit if you know what you are doing'
         required: false
+        type: string
+      token:
+        description: 'Personal Access Token'
+        required: true
+        default: ""
+        type: string
+  workflow_call: # Allows trigger the workflow from other workflow
+    inputs:
+      custom_version: # Optional input for a custom version
+        description: 'Custom version to publish (e.g., v1.2.3) -- only edit if you know what you are doing'
+        required: false
+        type: string
       token:
         description: 'Personal Access Token'
         required: true
         default: ""
         type: string
 
 jobs:
-  get-version:
-    uses: ./.github/workflows/get_semantic_release_version.yml
-    with:
-      custom_version: ${{ github.event.inputs.custom_version }}
-      token: ${{ github.event.inputs.token }}
-
   publish-python-sdk:
     uses: ./.github/workflows/publish_python_sdk.yml
+    secrets: inherit
     with:
       custom_version: ${{ github.event.inputs.custom_version }}
       token: ${{ github.event.inputs.token }}
 
   build-publish-docker-images:
     uses: ./.github/workflows/publish_images.yml
-    needs: [ get-version, publish-python-sdk ]
+    needs: [ publish-python-sdk ]
+    secrets: inherit
     with:
       custom_version: ${{ github.event.inputs.custom_version }}
       token: ${{ github.event.inputs.token }}
 
   publish-helm-charts:
     uses: ./.github/workflows/publish_helm_charts.yml
-    needs: [ get-version, publish-python-sdk ]
+    needs: [ publish-python-sdk ]
+    secrets: inherit
     with:
       custom_version: ${{ github.event.inputs.custom_version }}
       token: ${{ github.event.inputs.token }}
 
   publish-java-sdk:
     uses: ./.github/workflows/publish_java_sdk.yml
-    needs: [ get-version, publish-python-sdk ]
+    needs: [ publish-python-sdk ]
+    secrets: inherit
     with:
       custom_version: ${{ github.event.inputs.custom_version }}
       token: ${{ github.event.inputs.token }}