Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve documentation around Array API support #477

Merged
merged 1 commit into from
Jun 8, 2024
Merged

Improve documentation around Array API support #477

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Cubed is a distributed N-dimensional array library implemented in Python using b

- Implements the [Python Array API standard](https://data-apis.org/array-api/latest/) (see [coverage status](./api_status.md))
- Guaranteed maximum memory usage for standard array functions
- Follows [Dask Array](https://docs.dask.org/en/stable/array.html)'s chunked array API (`map_blocks`, `rechunk`, `apply_gufunc`, etc)
- Follows [Dask Array](https://docs.dask.org/en/stable/array.html)'s chunked array API (`map_blocks`, `map_overlap`, `rechunk`, `apply_gufunc`, etc)
- [Zarr](https://zarr.readthedocs.io/en/stable/) for persistent and intermediate storage
- Multiple serverless runtimes: Python (in-process), [Lithops](https://lithops-cloud.github.io/), [Modal](https://modal.com/), [Apache Beam](https://beam.apache.org/)
- Integration with [Xarray](https://xarray.dev/) via [cubed-xarray](https://github.com/xarray-contrib/cubed-xarray)
Expand Down
45 changes: 30 additions & 15 deletions api_status.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
## Array API Coverage Implementation Status

This table shows which parts of the the [Array API](https://data-apis.org/array-api/latest/API_specification/index.html) have been implemented in Cubed. For those that have not been implemented a rough level of difficulty is indicated (1=easy, 3=hard).
Cubed supports version [2022.12](https://data-apis.org/array-api/2022.12/index.html) of the Python array API standard, with a few exceptions noted below. The [linear algebra extensions](https://data-apis.org/array-api/2022.12/extensions/linear_algebra_functions.html) and [Fourier transform functions¶](https://data-apis.org/array-api/2022.12/extensions/fourier_transform_functions.html) are *not* supported.

| Category | Object/Function | Implemented | Difficulty | Notes |
Support for version [2023.12](https://data-apis.org/array-api/2023.12/index.html) is tracked in Cubed issue [#438](https://github.com/cubed-dev/cubed/issues/438).

This table shows which parts of the the [Array API](https://data-apis.org/array-api/latest/API_specification/index.html) have been implemented in Cubed, and which ones are missing. The version column shows the version when the feature was added to the standard, for version 2022.12 or later.

| Category | Object/Function | Implemented | Version | Notes |
| ------------------------ | ------------------- | ------------------ | ---------- | ---------------------------- |
| Array object | Arithmetic Ops | :white_check_mark: | | |
| | Array Ops | :white_check_mark: | | |
Expand Down Expand Up @@ -37,10 +41,16 @@ This table shows which parts of the the [Array API](https://data-apis.org/array-
| Data Types | `bool`, `int8`, ... | :white_check_mark: | | |
| Elementwise Functions | `add` | :white_check_mark: | | Example of a binary function |
| | `negative` | :white_check_mark: | | Example of a unary function |
| | _others_ | :white_check_mark: | | |
| | _others_ | :white_check_mark: | | Except 2023.12 functions in [#438](https://github.com/cubed-dev/cubed/issues/438) |
| Indexing | Single-axis | :white_check_mark: | | |
| | Multi-axis | :white_check_mark: | | |
| | Boolean array | :x: | 3 | Shape is data dependent, [#73](https://github.com/cubed-dev/cubed/issues/73) |
| | Boolean array | :x: | | Shape is data dependent, [#73](https://github.com/cubed-dev/cubed/issues/73) |
| Indexing Functions | `take` | :white_check_mark: | 2022.12 | |
| Inspection | `capabilities` | :x: | 2023.12 | |
| | `default_device` | :x: | 2023.12 | |
| | `default_dtypes` | :x: | 2023.12 | |
| | `devices` | :x: | 2023.12 | |
| | `dtypes` | :x: | 2023.12 | |
| Linear Algebra Functions | `matmul` | :white_check_mark: | | |
| | `matrix_transpose` | :white_check_mark: | | |
| | `tensordot` | :white_check_mark: | | |
Expand All @@ -49,28 +59,33 @@ This table shows which parts of the the [Array API](https://data-apis.org/array-
| | `broadcast_to` | :white_check_mark: | | |
| | `concat` | :white_check_mark: | | |
| | `expand_dims` | :white_check_mark: | | |
| | `flip` | :x: | 2 | Needs indexing with step=-1, [#114](https://github.com/cubed-dev/cubed/issues/114) |
| | `flip` | :x: | | Needs indexing with step=-1, [#114](https://github.com/cubed-dev/cubed/issues/114) |
| | `permute_dims` | :white_check_mark: | | |
| | `repeat` | :x: | 2023.12 | |
| | `reshape` | :white_check_mark: | | Partial implementation |
| | `roll` | :white_check_mark: | | |
| | `squeeze` | :white_check_mark: | | |
| | `stack` | :white_check_mark: | | |
| | `tile` | :x: | 2023.12 | |
| | `unstack` | :x: | 2023.12 | |
| Searching Functions | `argmax` | :white_check_mark: | | |
| | `argmin` | :white_check_mark: | | |
| | `nonzero` | :x: | 3 | Shape is data dependent |
| | `nonzero` | :x: | | Shape is data dependent |
| | `searchsorted` | :x: | 2023.12 | |
| | `where` | :white_check_mark: | | |
| Set Functions | `unique_all` | :x: | 3 | Shape is data dependent |
| | `unique_counts` | :x: | 3 | Shape is data dependent |
| | `unique_inverse` | :x: | 3 | Shape is data dependent |
| | `unique_values` | :x: | 3 | Shape is data dependent |
| Sorting Functions | `argsort` | :x: | 3 | Not in Dask |
| | `sort` | :x: | 3 | Not in Dask |
| Statistical Functions | `max` | :white_check_mark: | | |
| Set Functions | `unique_all` | :x: | | Shape is data dependent |
| | `unique_counts` | :x: | | Shape is data dependent |
| | `unique_inverse` | :x: | | Shape is data dependent |
| | `unique_values` | :x: | | Shape is data dependent |
| Sorting Functions | `argsort` | :x: | | Not in Dask |
| | `sort` | :x: | | Not in Dask |
| Statistical Functions | `cumulative_sum` | :x: | 2023.12 | |
| | `max` | :white_check_mark: | | |
| | `mean` | :white_check_mark: | | |
| | `min` | :white_check_mark: | | |
| | `prod` | :white_check_mark: | | |
| | `std` | :x: | 2 | Like `mean`, [#29](https://github.com/cubed-dev/cubed/issues/29) |
| | `std` | :x: | | Like `mean`, [#29](https://github.com/cubed-dev/cubed/issues/29) |
| | `sum` | :white_check_mark: | | |
| | `var` | :x: | 2 | Like `mean`, [#29](https://github.com/cubed-dev/cubed/issues/29) |
| | `var` | :x: | | Like `mean`, [#29](https://github.com/cubed-dev/cubed/issues/29) |
| Utility Functions | `all` | :white_check_mark: | | |
| | `any` | :white_check_mark: | | |
2 changes: 2 additions & 0 deletions docs/api.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ Chunk-specific functions
Non-standardised functions
==========================

These are functions that have not (yet) been included in the Python Array API Standard.

.. currentmodule:: cubed
.. autosummary::
:nosignatures:
Expand Down
9 changes: 6 additions & 3 deletions docs/array-api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
# Python Array API

Cubed implements the [Python Array API standard](https://data-apis.org/array-api/latest/) in `cubed.array_api`. Refer
to its [API specification](https://data-apis.org/array-api/latest/API_specification/index.html) for API documentation.
Cubed implements version 2022.12 of the [Python Array API standard](https://data-apis.org/array-api/2022.12/index.html) in `cubed.array_api`, with a few exceptions noted below. Refer to its [API specification](https://data-apis.org/array-api/2022.12/API_specification/index.html) for API documentation.

The [linear algebra extensions](https://data-apis.org/array-api/2022.12/extensions/linear_algebra_functions.html) and [Fourier transform functions¶](https://data-apis.org/array-api/2022.12/extensions/fourier_transform_functions.html) are *not* supported.

Support for version [2023.12](https://data-apis.org/array-api/2023.12/index.html) is tracked in Cubed issue [#438](https://github.com/cubed-dev/cubed/issues/438).

## Missing from Cubed

Expand All @@ -13,7 +16,6 @@ The following parts of the standard are not implemented:
| Creation Functions | `from_dlpack` |
| Indexing | Boolean array |
| Manipulation Functions | `flip` |
| | `roll` |
| Searching Functions | `nonzero` |
| Set Functions | `unique_all` |
| | `unique_counts` |
Expand Down Expand Up @@ -47,4 +49,5 @@ The following [Manipulation Functions](https://data-apis.org/array-api/latest/AP

```{eval-rst}
.. autofunction:: cubed.array_api.broadcast_to
.. autofunction:: cubed.array_api.concat
```
4 changes: 2 additions & 2 deletions docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Cubed is a distributed N-dimensional array library implemented in Python using b

- Implements the [Python Array API standard](https://data-apis.org/array-api/latest/)
- Guaranteed maximum memory usage for standard array functions
- Follows [Dask Array](https://docs.dask.org/en/stable/array.html)'s chunked array API (`map_blocks`, `rechunk`, etc)
- Follows [Dask Array](https://docs.dask.org/en/stable/array.html)'s chunked array API (`map_blocks`, `map_overlap`, `rechunk`, `apply_gufunc`, etc)
- [Zarr](https://zarr.readthedocs.io/en/stable/) for storage
- Multiple serverless runtimes: Python (in-process), [Lithops](https://lithops-cloud.github.io/), [Modal](https://modal.com/), [Apache Beam](https://beam.apache.org/)
- Integration with [Xarray](https://xarray.dev/) via [cubed-xarray](https://github.com/xarray-contrib/cubed-xarray)
Expand Down Expand Up @@ -42,4 +42,4 @@ contributing

[Cubed: Bounded-memory serverless array processing in xarray](https://xarray.dev/blog/cubed-xarray)

[Optimizing Cubed](https://medium.com/pangeo/optimizing-cubed-7a0b8f65f5b7)
[Optimizing Cubed](https://medium.com/pangeo/optimizing-cubed-7a0b8f65f5b7)
Loading