Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finish pyproject.toml migration #382

Merged
merged 46 commits into from
Nov 8, 2022
Merged
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
13b44c8
Gather system `import`s and separate from external
jakirkham Nov 3, 2022
b03845b
Reorder external `import`s
jakirkham Nov 3, 2022
7f41565
Run `import setuptools`
jakirkham Nov 3, 2022
aee9b9b
Use `setuptools` instead of `distutils`
jakirkham Nov 3, 2022
1833c83
Drop explicit `cythonize` calls
jakirkham Nov 3, 2022
2abad9f
Use Cython's `build_ext` (if available)
jakirkham Nov 3, 2022
e6f3a6a
Drop `setup_requires`
jakirkham Nov 3, 2022
296b0ac
Move setuptool-scm config to `pyproject.toml`
jakirkham Nov 3, 2022
f4a2537
Move nearly all of `setup.py` to `pyproject.toml`
jakirkham Nov 3, 2022
725c825
Add more URLs
jakirkham Nov 3, 2022
b735ef3
Add `.txt` extension to license
jakirkham Nov 3, 2022
6cf38ab
Use latest setuptools build backend
jakirkham Nov 3, 2022
445ca7d
Drop extra whitespace
jakirkham Nov 3, 2022
055167c
Add release note
jakirkham Nov 3, 2022
3f216d9
Fix lint
jakirkham Nov 3, 2022
798cd68
Move `pytest` config options to `pyproject.toml`
jakirkham Nov 3, 2022
f5f9b3f
Merge branch 'main' into update_bld
jakirkham Nov 4, 2022
2e48bec
Move `.coveragerc` into `pyproject.toml`
jakirkham Nov 4, 2022
e42a754
Merge zarr-developers/main into jakirkham/update_bld
jakirkham Nov 4, 2022
45fc1b3
Standardize on `setuptools-scm` with `-`
jakirkham Nov 4, 2022
1c57a80
Add `setuptools` & `setuptools-scm` to dev reqs
jakirkham Nov 4, 2022
916bf7a
Merge zarr-developers/main into jakirkham/update_bld
jakirkham Nov 4, 2022
05d40f6
Consolidate requirements
jakirkham Nov 4, 2022
72e9727
Move `setuptools` after `project` items
jakirkham Nov 4, 2022
e292f02
Simplify classifiers
jakirkham Nov 4, 2022
c14dda4
Merge branch 'main' into update_bld
jakirkham Nov 4, 2022
258d487
Merge branch 'main' into update_bld
jakirkham Nov 4, 2022
a4daba2
Drop doc build as it already happens on RTD
jakirkham Nov 4, 2022
b4bd856
Move `msgpack` later and unwrap
jakirkham Nov 4, 2022
541055c
Run `flake8` before tests
jakirkham Nov 4, 2022
d92014b
Bake coverage & doctest flags into pytest config
jakirkham Nov 4, 2022
9cc5f77
Merge zarr-developers/main into jakirkham/update_bld
jakirkham Nov 4, 2022
e8393c6
Drop `coveralls` from dependencies
jakirkham Nov 4, 2022
1d4500e
Fix Codecov release note reference
jakirkham Nov 4, 2022
9888271
Use `int16` in doctests for consistent reprs
jakirkham Nov 5, 2022
01072a8
Add coverage report output type to config too
jakirkham Nov 5, 2022
dcd3c6e
Push `numcodecs` test path into config
jakirkham Nov 5, 2022
0ced812
Exclude directories outside of numcodecs
jakirkham Nov 5, 2022
9e550c7
Simplify `flake8` instructions
jakirkham Nov 5, 2022
4175325
Delete unused `build.cmd`
jakirkham Nov 5, 2022
7f9fa08
Drop upper bound on Python
jakirkham Nov 5, 2022
c832d88
Drop `requirements_dev.txt` too
jakirkham Nov 5, 2022
e43e314
Merge branch 'main' into update_bld
jakirkham Nov 6, 2022
ed9d21d
Merge branch 'main' into update_bld
jakirkham Nov 7, 2022
6db7354
Merge branch 'main' into update_bld
jakirkham Nov 7, 2022
e679fb2
Always use Cython's `new_build_ext`
jakirkham Nov 8, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .coveragerc

This file was deleted.

12 changes: 12 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -1,2 +1,14 @@
[flake8]
max-line-length = 100
exclude =
.git,
.github,
.pytest_cache,
adhoc,
build,
c-blosc,
dist,
docs,
fixture,
notebooks,
numcodecs.egg-info,
15 changes: 4 additions & 11 deletions .github/workflows/ci-linux.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,23 +24,16 @@ jobs:

- name: Install numcodecs
run: |
python -m pip install -U pip -r requirements_test.txt -r requirements.txt
python -m pip install -v -e .
python -m pip install -v -e .[test,msgpack,zfpy]

- name: List installed packages
run: python -m pip list

- name: Run tests
run: pytest -v --cov=numcodecs --cov-report xml --doctest-modules --doctest-glob=*.pyx numcodecs

- name: Flake8
run: flake8 numcodecs
run: flake8

- name: Build Docs
run: |
pip install -r requirements_rtfd.txt
cd docs
sphinx-build -W -b html -d {envtmpdir}/doctrees . {envtmpdir}/html
- name: Run tests
run: pytest -v

- uses: codecov/codecov-action@v3
with:
Expand Down
6 changes: 2 additions & 4 deletions .github/workflows/ci-osx.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ jobs:
conda create -n env python=${{matrix.python-version}} wheel pip compilers 'clang>=12.0.1'
conda activate env
which pip
pip install -r requirements_test.txt -r requirements.txt
conda env export

- name: Show info about `env` environment
Expand All @@ -50,14 +49,13 @@ jobs:
run: |
conda activate env
export CC=clang
python -m pip install -U pip -r requirements_test.txt -r requirements.txt
python -m pip install -v -e .
python -m pip install -v -e .[test,msgpack,zfpy]

- name: Run tests
shell: "bash -l {0}"
run: |
conda activate env
pytest -v --cov=numcodecs --cov-report xml numcodecs
pytest -v

- uses: codecov/codecov-action@v3
with:
Expand Down
6 changes: 2 additions & 4 deletions .github/workflows/ci-windows.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,22 +30,20 @@ jobs:
conda create -n env python=${{matrix.python-version}} wheel pip compilers
conda activate env
which pip
pip install -r requirements_test.txt -r requirements.txt
conda env export

- name: Install numcodecs
shell: "bash -l {0}"
run: |
conda activate env
export CC=clang
python -m pip install -U pip -r requirements_test.txt -r requirements.txt
python -m pip install -v -e .
python -m pip install -v -e .[test,msgpack,zfpy]

- name: Run tests
shell: "bash -l {0}"
run: |
conda activate env
pytest -v --cov=numcodecs --cov-report xml numcodecs
pytest -v

- uses: codecov/codecov-action@v3
with:
Expand Down
14 changes: 0 additions & 14 deletions .pyup.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,3 @@
# see https://pyup.io/docs/configuration/ for all available options

schedule: every month

requirements:
- requirements.txt:
pin: False
update: False
- requirements_test.txt:
pin: False
update: False
- requirements_rtfd.txt:
pin: False
update: False
- requirements_dev.txt:
pin: True
update: all
3 changes: 2 additions & 1 deletion .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ sphinx:

python:
install:
- requirements: requirements_rtfd.txt
- method: pip
path: .
extra_requirements:
- docs
File renamed without changes.
47 changes: 0 additions & 47 deletions build.cmd

This file was deleted.

19 changes: 5 additions & 14 deletions docs/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -99,16 +99,11 @@ the repository, you can do something like the following::
$ mkdir -p ~/pyenv/numcodecs-dev
$ virtualenv --no-site-packages --python=/usr/bin/python3.9 ~/pyenv/numcodecs-dev
$ source ~/pyenv/numcodecs-dev/bin/activate
$ pip install -r requirements_dev.txt
$ python setup.py build_ext --inplace
$ pip install -e .[docs,test,msgpack,zfpy]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be "normal" to also have an "all" alias to cover these?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah agree that would be useful, but unless we write our own (which doesn't seem very DRY friendly) or handle this in setup.py. Am not aware of any existing option. Also please see this upstream issue ( pypa/pip#4340 ).


To verify that your development environment is working, you can run the unit tests::

$ pytest -v numcodecs

To install numcodecs globally in editable mode run::

$ python -m pip install -e .
$ pytest -v

Creating a branch
~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -147,11 +142,7 @@ Running the test suite
NumCodecs includes a suite of unit tests, as well as doctests included in function and class
docstrings. The simplest way to run the unit tests is to invoke::

$ pytest -v numcodecs

To also run the doctests within docstrings, run::

$ pytest -v --doctest-modules numcodecs
$ pytest -v

NumCodecs currently supports Python 6-3.9, so the above command must
succeed before code can be accepted into the main code base.
Expand All @@ -167,14 +158,14 @@ All code must conform to the PEP8 standard. Regarding line length, lines up to 1
characters are allowed, although please try to keep under 90 wherever possible.
Conformance can be checked by running::

$ flake8 --max-line-length=100 numcodecs
$ flake8

Test coverage
~~~~~~~~~~~~~

NumCodecs maintains 100% test coverage under the latest Python stable release (currently
Python 3.9). Both unit tests and docstring doctests are included when computing
coverage. Running ``pytest -v --cov=numcodecs`` will automatically run the test suite with coverage
coverage. Running ``pytest -v`` will automatically run the test suite with coverage
and produce a coverage report. This should be 100% before code can be accepted into the
main code base.

Expand Down
3 changes: 3 additions & 0 deletions docs/release.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@ Maintenance
* Add tests for all registry classes.
By :user:`Josh Moore <joshmoore>`, :issue:`349`.

* Finish ``pyproject.toml`` migration.
By :user:`John Kirkham <jakirkham>` :issue:`382`.

* Drop tox.
By :user:`John Kirkham <jakirkham>`, :issue:`384`.

Expand Down
4 changes: 2 additions & 2 deletions numcodecs/astype.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ class AsType(Codec):
>>> x = np.arange(100, 120, 2, dtype=np.int8)
>>> x
array([100, 102, 104, 106, 108, 110, 112, 114, 116, 118], dtype=int8)
>>> f = numcodecs.AsType(encode_dtype=x.dtype, decode_dtype=np.int64)
>>> f = numcodecs.AsType(encode_dtype=x.dtype, decode_dtype=np.int16)
>>> y = f.decode(x)
>>> y
array([100, 102, 104, 106, 108, 110, 112, 114, 116, 118])
array([100, 102, 104, 106, 108, 110, 112, 114, 116, 118], dtype=int16)
>>> z = f.encode(y)
>>> z
array([100, 102, 104, 106, 108, 110, 112, 114, 116, 118], dtype=int8)
Expand Down
6 changes: 3 additions & 3 deletions numcodecs/delta.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,14 +28,14 @@ class Delta(Codec):
--------
>>> import numcodecs
>>> import numpy as np
>>> x = np.arange(100, 120, 2, dtype='i8')
>>> codec = numcodecs.Delta(dtype='i8', astype='i1')
>>> x = np.arange(100, 120, 2, dtype='i2')
>>> codec = numcodecs.Delta(dtype='i2', astype='i1')
>>> y = codec.encode(x)
>>> y
array([100, 2, 2, 2, 2, 2, 2, 2, 2, 2], dtype=int8)
>>> z = codec.decode(y)
>>> z
array([100, 102, 104, 106, 108, 110, 112, 114, 116, 118])
array([100, 102, 104, 106, 108, 110, 112, 114, 116, 118], dtype=int16)

"""

Expand Down
11 changes: 7 additions & 4 deletions numcodecs/vlen.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -310,11 +310,14 @@ class VLenArray(Codec):
--------
>>> import numcodecs
>>> import numpy as np
>>> x = np.array([[1, 3, 5], [4], [7, 9]], dtype='object')
>>> codec = numcodecs.VLenArray('<i4')
>>> x1 = np.array([1, 3, 5], dtype=np.int16)
>>> x2 = np.array([4], dtype=np.int16)
>>> x3 = np.array([7, 9], dtype=np.int16)
>>> x = np.array([x1, x2, x3], dtype='object')
>>> codec = numcodecs.VLenArray('<i2')
>>> codec.decode(codec.encode(x))
array([array([1, 3, 5], dtype=int32), array([4], dtype=int32),
array([7, 9], dtype=int32)], dtype=object)
array([array([1, 3, 5], dtype=int16), array([4], dtype=int16),
array([7, 9], dtype=int16)], dtype=object)

See Also
--------
Expand Down
108 changes: 106 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,111 @@
[build-system]
requires = ["setuptools>=40.8.0", "setuptools_scm[toml]>=6.2", "Cython", "py-cpuinfo"]
build-backend = "setuptools.build_meta:__legacy__"
requires = [
"setuptools>=64",
"setuptools-scm[toml]>=6.2",
"Cython",
"py-cpuinfo"
]
build-backend = "setuptools.build_meta"

[project]
name = "numcodecs"
description = """
A Python package providing buffer compression and transformation codecs for use
in data storage and communication applications.
"""
readme = "README.rst"
dependencies = [
"entrypoints",
"numpy>=1.7",
"typing-extensions>=3.7.4"
]
requires-python = ">=3.7"
dynamic = [
"version",
]
classifiers = [
"Development Status :: 4 - Beta",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps outwith this PR but is "Beta" still appropriate?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. Filed as issue ( #394 )

"Intended Audience :: Developers",
"Intended Audience :: Information Technology",
"Intended Audience :: Science/Research",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python",
"Topic :: Software Development :: Libraries :: Python Modules",
"Operating System :: Unix",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3 :: Only",
]
maintainers = [
{ name = "Alistair Miles", email = "[email protected]" },
]
license = { text = "MIT" }

[project.urls]
"Bug Tracker" = "https://github.com/zarr-developers/numcodecs/issues"
Changelog = "https://numcodecs.readthedocs.io/en/stable/release.html"
Documentation = "https://numcodecs.readthedocs.io/"
Homepage = "https://github.com/zarr-developers/numcodecs"

[project.optional-dependencies]
docs = [
"sphinx",
"sphinx-issues",
"numpydoc",
"mock",
]
test = [
"coverage",
"flake8",
"pytest",
"pytest-cov",
]
msgpack = [
"msgpack",
]
zfpy = [
"zfpy>=1.0.0",
]

[tool.setuptools]
jakirkham marked this conversation as resolved.
Show resolved Hide resolved
license-files = ["LICENSE.txt"]
package-dir = {"" = "."}
packages = ["numcodecs", "numcodecs.tests"]
zip-safe = false

[tool.setuptools_scm]
version_scheme = "guess-next-dev"
local_scheme = "dirty-tag"
write_to = "numcodecs/version.py"

[tool.codespell]
skip = "./.git,fixture"
ignore-words-list = "ba, compiletime, hist, nd, unparseable"

[tool.coverage.report]
exclude_lines = [
"pragma: no cover",
"pragma: ${PY_MAJOR_VERSION} no cover",
]

[tool.pytest.ini_options]
addopts = "--cov=numcodecs --cov-report xml --doctest-modules --doctest-glob=*.pyx"
doctest_optionflags = [
"NORMALIZE_WHITESPACE",
"ELLIPSIS",
"IGNORE_EXCEPTION_DETAIL",
]
testpaths = [
"numcodecs",
]
norecursedirs = [
".git",
".github",
".pytest_cache",
"adhoc",
"build",
"c-blosc",
"docs",
"fixture",
"notebooks",
"numcodecs.egg-info",
]
2 changes: 0 additions & 2 deletions pytest.ini

This file was deleted.

Loading