Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable github action runners #1537

Merged
merged 12 commits into from
Jun 8, 2022
Merged

Enable github action runners #1537

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
99e595e
test github action runners (instead of self-hosted)
daxhaw May 20, 2022
4f682bc
move all workflows to run ubuntu-20.04 instead of self-hosted
daxhaw May 20, 2022
55c6a4f
update python code formatter
daxhaw May 20, 2022
f8685b1
Fixup python lint issues found by Black
daxhaw May 20, 2022
9527b6f
Cleanup requires sudo
daxhaw May 20, 2022
1b85941
use ssh-agent to fill in ssh env variables
daxhaw May 20, 2022
8e5e363
One more ssh-agent related break; streamline build for testing
daxhaw May 20, 2022
29887d0
fix typo in yaml
daxhaw May 20, 2022
707908a
Re-enable jobs in main; testing complete
daxhaw May 20, 2022
6ffe4ec
apply review feedback
daxhaw May 20, 2022
52e71d0
Merge branch 'master' into dax/test_github_runners
daxhaw Jun 8, 2022
6790791
did not mean to re-enable integration tests
daxhaw Jun 8, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/actions/build-job/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ runs:
- name: Build Docker Image
shell: bash
run: |
if pgrep ssh-agent; then pkill ssh-agent; fi
ssh-agent -a $SSH_AUTH_SOCK > /dev/null 2>&1
CFG_ENABLES=
if [ "${{ inputs.build-flavor }}" != "" ] ; then
CFG_ENABLES="$CFG_ENABLES --cfg-enables ${{ inputs.build-flavor}}"
Expand Down
2 changes: 1 addition & 1 deletion .github/actions/setup-job/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,6 @@ runs:
rm -rf /opt/ghc
echo "After Haskell Cleanup ~0.5G"
df -h
rm -rf /usr/local/lib/android
sudo rm -rf /usr/local/lib/android
echo "After Android Cleanup ~15G"
df -h
4 changes: 2 additions & 2 deletions .github/workflows/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ Each of our jobs is built on a standard Ubuntu:20.04 system.
To keep things simple, any specialization of jobs for different environments is done within the docker builds.

```YAML
runs-on: self-hosted
runs-on: ubuntu-20.04
```

### Environment Variables
Expand Down Expand Up @@ -299,7 +299,7 @@ This is required to pull the latest version from GitHub's image store:
```YAML
- name: Pull Latest Dev-Core Image
run: |
ssh-agent -a $SSH_AUTH_SOCK > /dev/null
ssh-agent -a $SSH_AUTH_SOCK > /dev/null 2>&1
echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
docker pull ghcr.io/gaia-platform/dev-base:latest
```
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/core.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ env:

jobs:
Core:
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand All @@ -29,7 +29,7 @@ jobs:
build-type: Release

Debug_Core:
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/cores-and-sdks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ env:

jobs:
Core:
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand All @@ -29,7 +29,7 @@ jobs:
build-type: Release

SDK:
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand All @@ -49,7 +49,7 @@ jobs:
build-type: Release

Debug_Core:
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand All @@ -69,7 +69,7 @@ jobs:
build-type: Debug

Debug_SDK:
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/documentation.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ env:
jobs:

Documentation:
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/integration-only.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ env:

jobs:
Integration_Tests:
runs-on: self-hosted
runs-on: ubuntu-20.04
timeout-minutes: 7
env:
GAIA_REPO: ${{ github.workspace }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/llvm-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ env:
jobs:

LLVM_Tests:
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand Down
31 changes: 16 additions & 15 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ env:
jobs:

Lint:
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand All @@ -31,7 +31,7 @@ jobs:
Core:
needs:
- Lint
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand All @@ -55,7 +55,7 @@ jobs:
needs:
- Lint
if: github.event_name != 'pull_request'
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand All @@ -78,7 +78,7 @@ jobs:
Debug_Core:
needs:
- Core
runs-on: self-hosted
runs-on: ubuntu-20.04
if: ${{ false }}
env:
GAIA_REPO: ${{ github.workspace }}
Expand All @@ -103,7 +103,7 @@ jobs:
needs:
- Core
if: github.event_name != 'pull_request'
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand All @@ -127,7 +127,7 @@ jobs:
- Lint
if: ${{ false }}
timeout-minutes: 200
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand All @@ -147,7 +147,7 @@ jobs:
needs:
- Lint
if: ${{ false }}
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand All @@ -170,9 +170,8 @@ jobs:
Integration_Tests:
needs:
- SDK
# if: github.event_name != 'pull_request'
if: ${{false}}
runs-on: self-hosted
if: github.event_name != 'pull_request'
runs-on: ubuntu-20.04
timeout-minutes: 7
env:
GAIA_REPO: ${{ github.workspace }}
Expand Down Expand Up @@ -217,7 +216,7 @@ jobs:
# Disabled due to inability to load libunwind.so.1 with Debug SDK.
# if: github.event_name != 'pull_request'
if: ${{ false }}
runs-on: self-hosted
runs-on: ubuntu-20.04
timeout-minutes: 7
env:
GAIA_REPO: ${{ github.workspace }}
Expand Down Expand Up @@ -256,7 +255,7 @@ jobs:
- Integration_Tests
# if: github.event_name != 'pull_request'
if: ${{false}}
runs-on: self-hosted
runs-on: ubuntu-20.04
timeout-minutes: 10
env:
GAIA_REPO: ${{ github.workspace }}
Expand Down Expand Up @@ -292,7 +291,7 @@ jobs:

Integration_Samples:
needs: SDK
runs-on: self-hosted
runs-on: ubuntu-20.04
if: github.event_name != 'pull_request'
env:
GAIA_REPO: ${{ github.workspace }}
Expand All @@ -314,6 +313,8 @@ jobs:
- name: Tests
working-directory: ${{ github.workspace }}
run: |
if pgrep ssh-agent; then pkill ssh-agent; fi
ssh-agent -a $SSH_AUTH_SOCK > /dev/null 2>&1
$GAIA_REPO/dev_tools/github-actions/execute_tests_against_package.sh \
--verbose \
--job-name $GITHUB_JOB \
Expand All @@ -332,7 +333,7 @@ jobs:
needs:
- Lint
if: ${{ false }}
runs-on: self-hosted
runs-on: ubuntu-20.04
env:
GAIA_REPO: ${{ github.workspace }}
steps:
Expand All @@ -357,7 +358,7 @@ jobs:
# - Performance_Tests
- Integration_Samples
# - Documentation
runs-on: self-hosted
runs-on: ubuntu-20.04
steps:
- uses: geekyeggo/delete-artifact@v1
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/performance-only.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ env:

jobs:
Performance_Tests:
runs-on: self-hosted
runs-on: ubuntu-20.04
timeout-minutes: 10
env:
GAIA_REPO: ${{ github.workspace }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/remove-old-artifacts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:

jobs:
remove-old-artifacts:
runs-on: self-hosted
runs-on: ubuntu-20.04
timeout-minutes: 5

steps:
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ repos:
# args: ["--severity=warning"] # Optionally only show errors and warnings

- repo: https://github.com/psf/black
rev: 21.7b0
rev: 22.3.0
hooks:
- id: black
exclude: ^(dev_tools/gdev/|production/tools/tests/gaiat/lit.cfg.py)
Expand Down
2 changes: 1 addition & 1 deletion dev_tools/docker_dev/gdev/sections/_abc/cfg.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ def __get_begin_pattern(self) -> Pattern:
Get the regex pattern that identifies the beginning of the section.
"""

begin_pattern = re.compile(fr"^\[{self.section_name}]$")
begin_pattern = re.compile(rf"^\[{self.section_name}]$")
self.log.debug("begin_pattern = %s", begin_pattern)
return begin_pattern

Expand Down
2 changes: 1 addition & 1 deletion dev_tools/docker_dev/gdev/sections/_abc/dockerfile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ def get_base_stages_text(self) -> str:
"""

base_stages_text = dedent(
fr"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this change for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This string prefix was flagged as a lint error when I had to upgrade our version of black. The old version of black on the github action runners had a dependency such that we couldn't run the python linter and the fix was to upgrade to the latest version. (change is in .pre-commit-config.yaml).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation! And what do these letters stand for, if they stand for anything?

Copy link
Contributor Author

@daxhaw daxhaw Jun 8, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They designate "format" f and "raw" r strings in python. Here is an explanation.

rf"""
#syntax=docker/dockerfile-upstream:master-experimental

# Static definition of base stages.
Expand Down
2 changes: 1 addition & 1 deletion dev_tools/docker_dev/gdev/sections/_custom/dockerfile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ def get_text(self) -> str:
login = os.getlogin()
text_parts.append(
dedent(
fr"""
rf"""
RUN groupadd -r -o -g {gid} {login} \
&& useradd {login} -l -r -o -u {uid} -g {gid} -G sudo \
&& mkdir -p {home} \
Expand Down