From 8b5414e39fb275b9d6269071d2c268339f99a47b Mon Sep 17 00:00:00 2001 From: David Hewitt <1939362+davidhewitt@users.noreply.github.com> Date: Thu, 4 Mar 2021 22:58:58 +0000 Subject: [PATCH] multiple-pymethods: documentation updates --- guide/src/class.md | 36 ++++++++++++++++++++---------------- guide/src/features.md | 8 ++++++++ guide/src/migration.md | 6 ++++++ src/lib.rs | 2 +- 4 files changed, 35 insertions(+), 17 deletions(-) diff --git a/guide/src/class.md b/guide/src/class.md index f53bc597db9..0ada756458d 100644 --- a/guide/src/class.md +++ b/guide/src/class.md @@ -1,8 +1,10 @@ # Python Classes -PyO3 exposes a group of attributes powered by Rust's proc macro system for defining Python classes as Rust structs. This chapter will discuss the functionality and configuration they offer. +PyO3 exposes a group of attributes powered by Rust's proc macro system for defining Python classes as Rust structs. -For ease of discovery, below is a list of all custom attributes with links to the relevant section of this chapter: +The main attribute is `#[pyclass]`, which is placed upon a Rust `struct` to generate a Python type for it. A struct will usually also have *one* `#[pymethods]`-annotated `impl` block for the struct, which is used to define Python methods and constants for the generated Python type. (If the [`multiple-pymethods`] feature is enabled each `#[pyclass]` is allowed to have multiple `#[pymethods]` blocks.) Finally, there may be multiple `#[pyproto]` trait implementations for the struct, which are used to define certain python magic methods such as `__str__`. + +This chapter will discuss the functionality and configuration these attributes offer. Below is a list of links to the relevant section of this chapter for each: - [`#[pyclass]`](#defining-a-new-class) - [`#[pyo3(get, set)]`](#object-properties-using-pyo3get-set) @@ -31,9 +33,9 @@ struct MyClass { } ``` -Because Python objects are freely shared between threads by the Python interpreter, all structs annotated with `#[pyclass]` must implement `Send`. +Because Python objects are freely shared between threads by the Python interpreter, all structs annotated with `#[pyclass]` must implement `Send` (unless annotated with [`#[pyclass(unsendable)]`](#customizing-the-class)). -The above example generates implementations for [`PyTypeInfo`], [`PyTypeObject`], and [`PyClass`] for `MyClass`. To see these generated implementations, refer to the section [How methods are implemented](#how-methods-are-implemented) at the end of this chapter. +The above example generates implementations for [`PyTypeInfo`], [`PyTypeObject`], and [`PyClass`] for `MyClass`. To see these generated implementations, refer to the [implementation details](#implementation-details) at the end of this chapter. ## Adding the class to a module @@ -444,7 +446,8 @@ To define a Python compatible method, an `impl` block for your struct has to be block with some variations, like descriptors, class method static methods, etc. Since Rust allows any number of `impl` blocks, you can easily split methods -between those accessible to Python (and Rust) and those accessible only to Rust. +between those accessible to Python (and Rust) and those accessible only to Rust. However to have multiple +`#[pymethods]`-annotated `impl` blocks for the same struct you must enable the [`multiple-pymethods`] feature of PyO3. ```rust # use pyo3::prelude::*; @@ -698,22 +701,21 @@ num=44, debug=false num=-1, debug=false ``` -## How methods are implemented +## Implementation details + +The `#[pyclass]` macros rely on a lot of conditional code generation: each `#[pyclass]` can optionally have a `#[pymethods]` block as well as several different possible `#[pyproto]` trait implementations. + +To support this flexibility the `#[pyclass]` macro expands to a blob of boilerplate code which sets up the structure for ["dtolnay specialization"](https://github.com/dtolnay/case-studies/blob/master/autoref-specialization/README.md). This implementation pattern enables the Rust compiler to use `#[pymethods]` and `#[pyproto]` implementations when they are present, and fall back to default (empty) definitions when they are not. -Users should be able to define a `#[pyclass]` with or without `#[pymethods]`, while PyO3 needs a -trait with a function that returns all methods. Since it's impossible to make the code generation in -pyclass dependent on whether there is an impl block, we'd need to implement the trait on -`#[pyclass]` and override the implementation in `#[pymethods]`. -To enable this, we use a static registry type provided by [inventory](https://github.com/dtolnay/inventory), -which allows us to collect `impl`s from arbitrary source code by exploiting some binary trick. -See [inventory: how it works](https://github.com/dtolnay/inventory#how-it-works) and `pyo3_macros_backend::py_class` for more details. +This simple technique works for the case when there is zero or one implementations. To support multiple `#[pymethods]` for a `#[pyclass]` (in the [`multiple-pymethods`] feature), a registry mechanism provided by the [`inventory`](https://github.com/dtolnay/inventory) crate is used instead. This collects `impl`s at library load time, but isn't supported on all platforms. See [inventory: how it works](https://github.com/dtolnay/inventory#how-it-works) for more details. -Specifically, the following implementation is generated: +The `#[pyclass]` macro expands to roughly the code seen below. The `PyClassImplCollector` is the type used internally by PyO3 for dtolnay specialization: ```rust # #[cfg(not(feature = "multiple-pymethods"))] -# { // The implementation differs slightly with the multiple-pymethods feature -use pyo3::prelude::*; +# { +# use pyo3::prelude::*; +// Note: the implementation differs slightly with the `multiple-pymethods` feature enabled. /// Class for demonstration struct MyClass { @@ -826,3 +828,5 @@ impl pyo3::class::impl_::PyClassImpl for MyClass { [`RefCell`]: https://doc.rust-lang.org/std/cell/struct.RefCell.html [classattr]: https://docs.python.org/3/tutorial/classes.html#class-and-instance-variables + +[`multiple-pymethods`]: features.md#multiple-pymethods diff --git a/guide/src/features.md b/guide/src/features.md index 315532ad8ca..ec1ec3cc58f 100644 --- a/guide/src/features.md +++ b/guide/src/features.md @@ -55,6 +55,14 @@ These macros require a number of dependencies which may not be needed by users w > This feature is enabled by default. To disable it, set `default-features = false` for the `pyo3` entry in your Cargo.toml. +### `multiple-pymethods` + +This feature enables a dependency on `inventory`, which enables each `#[pyclass]` to have more than one `#[pymethods]` block. + +Most users should only need a single `#[pymethods]` per `#[pyclass]`. In addition, not all platforms (e.g. Wasm) are supported by `inventory`. For this reason this feature is not enabled by default, meaning fewer dependencies and faster compilation for the majority of users. + +See [the `#[pyclass]` implementation details](class.md#implementation-details) for more information. + ### `nightly` The `nightly` feature needs the nightly Rust compiler. This allows PyO3 to use Rust's unstable specialization feature to apply the following optimizations: diff --git a/guide/src/migration.md b/guide/src/migration.md index 9e2ae2ba99c..cafba539b22 100644 --- a/guide/src/migration.md +++ b/guide/src/migration.md @@ -9,6 +9,12 @@ For a detailed list of all changes, see the [CHANGELOG](changelog.md). For projects embedding Python in Rust, PyO3 no longer automatically initalizes a Python interpreter on the first call to `Python::with_gil` (or `Python::acquire_gil`) unless the [`auto-initalize` feature](features.md#auto-initalize) is enabled. +### New `multiple-pymethods` feature + +`#[pymethods]` have been reworked with a simpler default implementation which removes the dependency on the `inventory` crate. This reduces dependencies and compile times for the majority of users. + +The limitation of the new default implementation is that it cannot support multiple `#[pymethods]` blocks for the same `#[pyclass]`. If you need this functionality, you must enable the `multiple-pymethods` feature which will switch `#[pymethods]` to the inventory-based implementation. + ## from 0.12.* to 0.13 ### Minimum Rust version increased to Rust 1.45 diff --git a/src/lib.rs b/src/lib.rs index 5d357cb729d..e4520c608a4 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -172,7 +172,7 @@ pub use { }; #[cfg(all(feature = "macros", feature = "multiple-pymethods"))] -pub use inventory; +pub use inventory; // Re-exported for `#[pyclass]` and `#[pymethods]` with `multiple-pymethods`. #[macro_use] mod internal_tricks;