Skip to content
forked from pydata/xarray

Commit

Permalink
Merge branch 'main' into groupby-shuffle
Browse files Browse the repository at this point in the history
* main:
  Enforce ruff/flake8-pie rules (PIE) (pydata#9740)
  Enforce ruff/flake8-comprehensions rules (C4) (pydata#9724)
  Enforce ruff/Perflint rules (PERF)  (pydata#9730)
  Apply ruff rule RUF007 (pydata#9739)
  chmod -x (pydata#9725)
  Aplpy ruff rules (RUF) (pydata#9731)
  Fix typos found by codespell (pydata#9721)
  support for additional scipy nd interpolants  (pydata#9599)
  Apply ruff/flake8-simplify rules (SIM) (pydata#9727)
  Apply ruff/flake8-implicit-str-concat rules (ISC) (pydata#9722)
  Apply ruff/flake8-pie rules (PIE) (pydata#9726)
  Enforce ruff/pygrep-hooks rules (PGH) (pydata#9729)
  Move to micromamba 2 (pydata#9732)
  Fix groupby tests (pydata#9716)
  Add missing xarray.core.missing import (pydata#9714)
  Fix writing of DataTree subgroups to zarr or netCDF (pydata#9677)
  Bump pypa/gh-action-pypi-publish from 1.10.3 to 1.11.0 in the actions group (pydata#9707)
  Update pre-commit hooks (pydata#9713)
  • Loading branch information
dcherian committed Nov 7, 2024
2 parents b100fb1 + c65b8a6 commit 978fad9
Show file tree
Hide file tree
Showing 85 changed files with 779 additions and 615 deletions.
1 change: 1 addition & 0 deletions .github/workflows/benchmarks-last-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ jobs:
- name: Set up conda environment
uses: mamba-org/setup-micromamba@v2
with:
micromamba-version: '2.0.2-2'
environment-file: ${{env.CONDA_ENV_FILE}}
environment-name: xarray-tests
cache-environment: true
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/benchmarks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ jobs:
- name: Set up conda environment
uses: mamba-org/setup-micromamba@v2
with:
micromamba-version: '2.0.2-2'
environment-file: ${{env.CONDA_ENV_FILE}}
environment-name: xarray-tests
cache-environment: true
Expand Down
27 changes: 10 additions & 17 deletions .github/workflows/ci-additional.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ concurrency:

env:
FORCE_COLOR: 3
MICROMAMBA_VERSION: "2.0.2-2"

jobs:
detect-ci-trigger:
Expand Down Expand Up @@ -57,11 +58,11 @@ jobs:
- name: Setup micromamba
uses: mamba-org/setup-micromamba@v2
with:
micromamba-version: ${{env.MICROMAMBA_VERSION}}
environment-file: ${{env.CONDA_ENV_FILE}}
environment-name: xarray-tests
create-args: >-
python=${{env.PYTHON_VERSION}}
conda
cache-environment: true
cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-${{env.TODAY}}-${{hashFiles(env.CONDA_ENV_FILE)}}"

Expand All @@ -70,8 +71,6 @@ jobs:
python -m pip install --no-deps -e .
- name: Version info
run: |
conda info -a
conda list
python xarray/util/print_versions.py
- name: Run doctests
run: |
Expand Down Expand Up @@ -105,20 +104,18 @@ jobs:
- name: Setup micromamba
uses: mamba-org/setup-micromamba@v2
with:
micromamba-version: ${{env.MICROMAMBA_VERSION}}
environment-file: ${{env.CONDA_ENV_FILE}}
environment-name: xarray-tests
create-args: >-
python=${{env.PYTHON_VERSION}}
conda
cache-environment: true
cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-${{env.TODAY}}-${{hashFiles(env.CONDA_ENV_FILE)}}"
- name: Install xarray
run: |
python -m pip install --no-deps -e .
- name: Version info
run: |
conda info -a
conda list
python xarray/util/print_versions.py
- name: Install mypy
run: |
Expand Down Expand Up @@ -159,20 +156,18 @@ jobs:
- name: Setup micromamba
uses: mamba-org/setup-micromamba@v2
with:
micromamba-version: ${{env.MICROMAMBA_VERSION}}
environment-file: ${{env.CONDA_ENV_FILE}}
environment-name: xarray-tests
create-args: >-
python=${{env.PYTHON_VERSION}}
conda
cache-environment: true
cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-${{env.TODAY}}-${{hashFiles(env.CONDA_ENV_FILE)}}"
- name: Install xarray
run: |
python -m pip install --no-deps -e .
- name: Version info
run: |
conda info -a
conda list
python xarray/util/print_versions.py
- name: Install mypy
run: |
Expand Down Expand Up @@ -218,20 +213,18 @@ jobs:
- name: Setup micromamba
uses: mamba-org/setup-micromamba@v2
with:
micromamba-version: ${{env.MICROMAMBA_VERSION}}
environment-file: ${{env.CONDA_ENV_FILE}}
environment-name: xarray-tests
create-args: >-
python=${{env.PYTHON_VERSION}}
conda
cache-environment: true
cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-${{env.TODAY}}-${{hashFiles(env.CONDA_ENV_FILE)}}"
- name: Install xarray
run: |
python -m pip install --no-deps -e .
- name: Version info
run: |
conda info -a
conda list
python xarray/util/print_versions.py
- name: Install pyright
run: |
Expand Down Expand Up @@ -277,20 +270,18 @@ jobs:
- name: Setup micromamba
uses: mamba-org/setup-micromamba@v2
with:
micromamba-version: ${{env.MICROMAMBA_VERSION}}
environment-file: ${{env.CONDA_ENV_FILE}}
environment-name: xarray-tests
create-args: >-
python=${{env.PYTHON_VERSION}}
conda
cache-environment: true
cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-${{env.TODAY}}-${{hashFiles(env.CONDA_ENV_FILE)}}"
- name: Install xarray
run: |
python -m pip install --no-deps -e .
- name: Version info
run: |
conda info -a
conda list
python xarray/util/print_versions.py
- name: Install pyright
run: |
Expand All @@ -313,7 +304,8 @@ jobs:
name: Minimum Version Policy
runs-on: "ubuntu-latest"
needs: detect-ci-trigger
if: needs.detect-ci-trigger.outputs.triggered == 'false'
# disabled until `conda` is compatible with the new `libmambapy`
if: false && needs.detect-ci-trigger.outputs.triggered == 'false'
defaults:
run:
shell: bash -l {0}
Expand All @@ -326,12 +318,13 @@ jobs:
- name: Setup micromamba
uses: mamba-org/setup-micromamba@v2
with:
micromamba-version: ${{env.MICROMAMBA_VERSION}}
environment-name: xarray-tests
create-args: >-
python=3.12
pyyaml
conda
python-dateutil
libmambapy
- name: All-deps minimum versions policy
run: |
Expand Down
5 changes: 2 additions & 3 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ concurrency:

env:
FORCE_COLOR: 3
micromamba_version: 2

jobs:
detect-ci-trigger:
Expand Down Expand Up @@ -110,13 +111,13 @@ jobs:
- name: Setup micromamba
uses: mamba-org/setup-micromamba@v2
with:
micromamba-version: '2.0.2-2'
environment-file: ${{ env.CONDA_ENV_FILE }}
environment-name: xarray-tests
cache-environment: true
cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{matrix.python-version}}-${{env.TODAY}}-${{hashFiles(env.CONDA_ENV_FILE)}}"
create-args: >-
python=${{matrix.python-version}}
conda
# We only want to install this on one run, because otherwise we'll have
# duplicate annotations.
Expand All @@ -131,8 +132,6 @@ jobs:
- name: Version info
run: |
conda info -a
conda list
python xarray/util/print_versions.py
- name: Import xarray
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/hypothesis.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ jobs:
- name: Setup micromamba
uses: mamba-org/setup-micromamba@v2
with:
micromamba-version: "2.0.2-2"
environment-file: ci/requirements/environment.yml
environment-name: xarray-tests
create-args: >-
Expand All @@ -76,8 +77,6 @@ jobs:
python -m pip install --no-deps -e .
- name: Version info
run: |
conda info -a
conda list
python xarray/util/print_versions.py
# https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/pypi-release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ jobs:
path: dist
- name: Publish package to TestPyPI
if: github.event_name == 'push'
uses: pypa/gh-action-pypi-publish@v1.10.3
uses: pypa/gh-action-pypi-publish@v1.11.0
with:
repository_url: https://test.pypi.org/legacy/
verbose: true
Expand All @@ -111,6 +111,6 @@ jobs:
name: releases
path: dist
- name: Publish package to PyPI
uses: pypa/gh-action-pypi-publish@v1.10.3
uses: pypa/gh-action-pypi-publish@v1.11.0
with:
verbose: true
9 changes: 3 additions & 6 deletions .github/workflows/upstream-dev-ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ concurrency:

env:
FORCE_COLOR: 3
MICROMAMBA_VERSION: "2.0.2-2"

jobs:
detect-ci-trigger:
Expand Down Expand Up @@ -63,12 +64,12 @@ jobs:
- name: Set up conda environment
uses: mamba-org/setup-micromamba@v2
with:
micromamba-version: ${{env.MICROMAMBA_VERSION}}
environment-file: ci/requirements/environment.yml
environment-name: xarray-tests
create-args: >-
python=${{ matrix.python-version }}
pytest-reportlog
conda
- name: Install upstream versions
run: |
bash ci/install-upstream-wheels.sh
Expand All @@ -77,8 +78,6 @@ jobs:
python -m pip install --no-deps -e .
- name: Version info
run: |
conda info -a
conda list
python xarray/util/print_versions.py
- name: Import xarray
run: |
Expand Down Expand Up @@ -122,12 +121,12 @@ jobs:
- name: Set up conda environment
uses: mamba-org/setup-micromamba@v2
with:
micromamba-version: ${{env.MICROMAMBA_VERSION}}
environment-file: ci/requirements/environment.yml
environment-name: xarray-tests
create-args: >-
python=${{ matrix.python-version }}
pytest-reportlog
conda
- name: Install upstream versions
run: |
bash ci/install-upstream-wheels.sh
Expand All @@ -136,8 +135,6 @@ jobs:
python -m pip install --no-deps -e .
- name: Version info
run: |
conda info -a
conda list
python xarray/util/print_versions.py
- name: Install mypy
run: |
Expand Down
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ repos:
- id: mixed-line-ending
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: 'v0.6.9'
rev: 'v0.7.2'
hooks:
- id: ruff-format
- id: ruff
Expand All @@ -25,7 +25,7 @@ repos:
exclude: "generate_aggregations.py"
additional_dependencies: ["black==24.8.0"]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.11.2
rev: v1.13.0
hooks:
- id: mypy
# Copied from setup.cfg
Expand Down
2 changes: 1 addition & 1 deletion DATATREE_MIGRATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ A number of other API changes have been made, which should only require minor mo
- The method `DataTree.to_dataset()` still exists but now has different options for controlling which variables are present on the resulting `Dataset`, e.g. `inherit=True/False`.
- `DataTree.copy()` also has a new `inherit` keyword argument for controlling whether or not coordinates defined on parents are copied (only relevant when copying a non-root node).
- The `DataTree.parent` property is now read-only. To assign a ancestral relationships directly you must instead use the `.children` property on the parent node, which remains settable.
- Similarly the `parent` kwarg has been removed from the `DataTree.__init__` constuctor.
- Similarly the `parent` kwarg has been removed from the `DataTree.__init__` constructor.
- DataTree objects passed to the `children` kwarg in `DataTree.__init__` are now shallow-copied.
- `DataTree.as_array` has been replaced by `DataTree.to_dataarray`.
- A number of methods which were not well tested have been (temporarily) disabled. In general we have tried to only keep things that are known to work, with the plan to increase API surface incrementally after release.
Expand Down
4 changes: 2 additions & 2 deletions asv_bench/benchmarks/dataset_io.py
Original file line number Diff line number Diff line change
Expand Up @@ -712,7 +712,7 @@ def load(self) -> tuple:
dims=("time",),
fastpath=True,
)
for v in range(0, n_variables)
for v in range(n_variables)
}
attributes = {}

Expand All @@ -722,7 +722,7 @@ class PerformanceBackend(xr.backends.BackendEntrypoint):
def open_dataset(
self,
filename_or_obj: str | os.PathLike | None,
drop_variables: tuple[str, ...] = None,
drop_variables: tuple[str, ...] | None = None,
*,
mask_and_scale=True,
decode_times=True,
Expand Down
2 changes: 1 addition & 1 deletion asv_bench/benchmarks/groupby.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# import flox to avoid the cost of first import
import cftime
import flox.xarray # noqa
import flox.xarray # noqa: F401
import numpy as np
import pandas as pd

Expand Down
8 changes: 4 additions & 4 deletions ci/min_deps_check.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
from collections.abc import Iterator
from datetime import datetime

import conda.api # type: ignore[import]
import libmambapy # type: ignore[import]
import yaml
from dateutil.relativedelta import relativedelta

Expand Down Expand Up @@ -62,7 +62,7 @@ def parse_requirements(fname) -> Iterator[tuple[str, int, int, int | None]]:
pkg, eq, version = row.partition("=")
if pkg.rstrip("<>") in IGNORE_DEPS:
continue
if pkg.endswith("<") or pkg.endswith(">") or eq != "=":
if pkg.endswith(("<", ">")) or eq != "=":
error("package should be pinned with exact version: " + row)
continue

Expand Down Expand Up @@ -93,7 +93,7 @@ def metadata(entry):

return (major, minor), time

raw_data = conda.api.SubdirData.query_all(pkg, channels=CHANNELS)
raw_data = libmambapy.SubdirData.query_all(pkg, channels=CHANNELS)
data = sorted(metadata(entry) for entry in raw_data if entry.timestamp != 0)

release_dates = {
Expand Down Expand Up @@ -186,7 +186,7 @@ def process_pkg(
)


def fmt_version(major: int, minor: int, patch: int = None) -> str:
def fmt_version(major: int, minor: int, patch: int | None = None) -> str:
if patch is None:
return f"{major}.{minor}"
else:
Expand Down
2 changes: 1 addition & 1 deletion doc/getting-started-guide/quick-overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,7 @@ contain another two subgroups, named ``fine`` and ``coarse``.
The (sub)subgroups ``fine`` and ``coarse`` contain two very similar datasets. They both have an ``"x"``
dimension, but the dimension is of different lengths in each group, which makes the data in each group
unalignable. In the root group we placed some completely unrelated information, in order to show how a tree can
store heterogenous data.
store heterogeneous data.

Remember to keep unalignable dimensions in sibling groups because a DataTree inherits coordinates down through its
child nodes. You can see this inheritance in the above representation of the DataTree. The coordinates
Expand Down
6 changes: 3 additions & 3 deletions doc/user-guide/data-structures.rst
Original file line number Diff line number Diff line change
Expand Up @@ -609,7 +609,7 @@ We have created a tree with three nodes in it:
Consistency checks are enforced. For instance, if we try to create a `cycle`,
where the root node is also a child of a decendent, the constructor will raise
where the root node is also a child of a descendant, the constructor will raise
an (:py:class:`~xarray.InvalidTreeError`):

.. ipython:: python
Expand Down Expand Up @@ -711,8 +711,8 @@ inherited dimensions, but DataTree's inheritance is slightly stricter yet
easier to reason about.

The constraint that this puts on a DataTree is that dimensions and indices that
are inherited must be aligned with any direct decendent node's existing
dimension or index. This allows decendents to use dimensions defined in
are inherited must be aligned with any direct descendant node's existing
dimension or index. This allows descendants to use dimensions defined in
ancestor nodes, without duplicating that information. But as a consequence, if
a dimension-name is defined in on a node and that same dimension-name
exists in one of its ancestors, they must align (have the same index and
Expand Down
Loading

0 comments on commit 978fad9

Please sign in to comment.