Support consuming PyDiGraph objects in Rust backends for Python libraries

[lochsh]

Motivation

I have had a read of #108 and #459, and I wonder if a use case has been overlooked. It's possible I just haven't realised how to achieve my use case.

I have a Python library that uses qiskit. I'm exploring the option of moving much of the implementation to Rust while maintaining a Python interface, using PyO3 to create the appropriate bindings.

I would have thought that I'd want my Rust library to depend on rustworkx, not rustworkx_core for this.

The Python frontend would create the qiskit.dagcircuit.DAGCircuit object. I'd like the Rust backend to be able to consume this. One option seemed to be creating a PyDiGraph from the DAGCircuit (it seems one already exists as the _multi_graph member on DAGCircuit) and then having that be consumed by the Rust backend. Given this is already a Rust-defined class that PyO3 uses to generate the Python class (aiui), I would have thought I could then use rustworkx to transform the data in that struct, and return some new data to the Python front-end.

Current behaviour

I can cargo add retworkx but doing use retworkx; in my Rust library gives me a

error[E0432]: unresolved import `retworkx`
 --> src/lib.rs:2:5
  |
2 | use retworkx;
  |     ^^^^^^^^ no external crate `retworkx`

as I think there isn't a public interface exported for me to use.

What is the expected enhancement?

Being able to build Rust libraries to be used as backends to Python tools that can utilise rustworkx.

[lochsh]

I think the issue is just that only a cdylib is built, and no rust lib. Perhaps the crate-type = ["cydlib"] line in the Cargo.toml could be updated?

[mtreinish]

This one isn't as easy to solve as it might appear. This is basically a limit of how interfaces to python need to be built, You're pretty much correct it's because we build a cdylib as the build artifact, basically for a pyo3 extension our primary output is a dynamic library with a C api that python knows how to talk to. Rust code though doesn't know how to link against or talk to that python interface by itself so even if we built both a cdylib and an rlib the rust code trying to share a PyDiGraph between rustworkx and your custom crate wouldn't be able to know they're the same thing. This use case is basically the same as what is being tracked in: PyO3/pyo3#1444

I'm actually hitting the same basic issue in qiskit-terra myself right now with: Qiskit/qiskit#8388 where I needed 2 graph objects in qiskit terra's downstream rust crate that are from PyDiGraphs. In that case I just copied/re-created new graph objects as a petgraph struct in qiskit's crate directly and used retworkx-core functions as needed on those objects.

The only solution I've come up with for sharing objects between crates (besides working towards a fix in PyO3/pyo3#1444 ) is to manually build a C FFI for rustworkx that we can then manually build a rust crate that wraps that C API and knows how to work with it (which is being tracked in #33). But this isn't easy either and is a lot of work, which is why it's the oldest open issue on the library. :)