Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save minutes by avoiding duplicate test runs #150

Merged
merged 10 commits into from
Mar 26, 2024
49 changes: 24 additions & 25 deletions .github/workflows/pytest.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
name: Test

on:
push:
branches: [ main ]
pull_request:
branches: [ main , "migrate-*"]
schedule:
Expand All @@ -18,42 +16,41 @@ jobs:
pytest:
strategy:
matrix:
# One job per OS; latest python version testable on GitHub actions.
# These should match the versions used in the "pytest" workflows of both
# ixmp and message_ix.
# One job per OS.
version:
- { os: macos-latest, python: "3.11" }
- { os: ubuntu-latest, python: "3.11" }
- { os: windows-latest, python: "3.11" }
# Versions of both ixmp and message_ix to use
- { os: macos-latest }
- { os: ubuntu-latest }
- { os: windows-latest }
# Versions of both ixmp and message_ix to use plus latest supported python version
upstream:
- { version: v3.4.0, extra-deps: '"pandas<2.0"' } # Minimum version given in setup.cfg
- { version: v3.5.0, extra-deps: '"pandas<2.0"' }
- { version: v3.6.0, extra-deps: '"pandas<2.0"' }
- { version: v3.7.0, extra-deps: "" } # Latest released version
- { version: main, extra-deps: "" } # Development version
- { version: v3.4.0, extra-deps: '"pandas<2.0"', python-version: "3.11" } # Minimum version given in pyproject.toml
- { version: v3.5.0, extra-deps: '"pandas<2.0"', python-version: "3.11" }
- { version: v3.6.0, extra-deps: '"pandas<2.0"', python-version: "3.11" }
- { version: v3.7.0, extra-deps: "", python-version: "3.11" }
- { version: v3.8.0, extra-deps: "", python-version: "3.12" } # Latest released version
- { version: main, extra-deps: "", python-version: "3.12" } # Development version

fail-fast: false

runs-on: ${{ matrix.version.os }}
name: ${{ matrix.version.os }}-py${{ matrix.version.python }}-upstream-${{ matrix.upstream.version }}
name: ${{ matrix.version.os }}-py${{ matrix.upstream.python-version }}-upstream-${{ matrix.upstream.version }}

steps:
- name: Cache test data
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: message-local-data
key: ${{ matrix.version.os }}

- name: Check out message-ix-models
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
lfs: true
fetch-depth: ${{ env.depth }}

- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.version.python }}
python-version: ${{ matrix.upstream.python-version }}
cache: pip
cache-dependency-path: "**/pyproject.toml"

Expand All @@ -62,7 +59,7 @@ jobs:
version: 25.1.1
license: ${{ secrets.GAMS_LICENSE }}

- uses: ts-graphviz/setup-graphviz@v1
- uses: ts-graphviz/setup-graphviz@v2
with:
macos-skip-brew-update: true

Expand Down Expand Up @@ -103,21 +100,23 @@ jobs:
shell: bash

- name: Upload test coverage to Codecov.io
uses: codecov/codecov-action@v3
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }} # required

pre-commit:
name: Code quality

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with: { python-version: "3.11" }
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12" }

- name: Force recreation of pre-commit virtual environment for mypy
if: github.event_name == 'schedule' # Comment this line to run on a PR
run: gh cache list -L 999 | cut -f2 | grep pre-commit | xargs -I{} gh cache delete "{}" || true
env: { GH_TOKEN: "${{ github.token }}" }

- uses: pre-commit/[email protected].0
- uses: pre-commit/[email protected].1
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ repos:
pass_filenames: false

language: python
entry: bash -c ". ${PRE_COMMIT_MYPY_VENV:-/dev/null}/bin/activate 2>/dev/null; mypy $0 $@"
entry: bash -c ". ${PRE_COMMIT_MYPY_VENV:-/dev/null}/bin/activate 2>/dev/null; mypy $0 $@; python -m pip list"
additional_dependencies:
- mypy >= 1.8.0
- plotnine
Expand Down
9 changes: 5 additions & 4 deletions message_ix_models/tests/test_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -156,10 +156,11 @@ def test_convert_units(recwarn):

# With store="quantity", a series of pint.Quantity is returned
result = convert_units(*args, store="quantity")
assert all(
np.isclose(a, b, atol=1e-4 * registry.kg)
for a, b in zip(exp.values, result.values)
)
# Will raise a DimensionalityError if units are not equal
ratios = [(a / b) for a, b in zip(exp.values, result.values)]
# Assert equal units and sufficiently close values
for ratio in ratios:
assert ratio.dimensionless and np.isclose(ratio, 1, atol=1e-4)

# With store="magnitude", a series of floats
exp = pd.Series([q.magnitude for q in exp.values], name="bar")
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ classifiers = [
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: R",
"Topic :: Scientific/Engineering",
"Topic :: Scientific/Engineering :: Information Analysis",
Expand Down
Loading