Subclassing pyclass from python - new returns wrong type

[davidhewitt]

Originally posted by @thesketh in #945 (comment)

I am using #[pyclass(subclass)], but ParentClass.__new__ seems to ignore the first argument and always returns an instance of the parent class instead. The only thing it seems I can do with subclassing is create new constructors

This needs investigating, sounds like a possible bug.

[thesketh]

A minimum example which reproduces this:

On the Rust side:

use pyo3::prelude::*;

#[pyclass(subclass)]
struct PyClass {}

#[pymethods]
impl PyClass {
    #[new]
    fn new() -> Self {
        Self{}
    }

    fn get_source(&self) -> PyResult<&'static str> {
        Ok("Rust")
    }
}

#[pymodule]
fn example(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_class::<PyClass>()?;
    Ok(())
}

On the Python side:

from example import PyClass

class Class(PyClass):
    def __new__(cls):
        return super().__new__(cls)  # This should return an instance of Class

    def get_source(self):
        return "Python"

a = PyClass()
# Should print "Rust":
print(a.get_source())
# Rust
b = Class()
# Should print "Python":
print(b.get_source())
# Rust 

[kngwyu]

I'm not sure we can fix this 🤔
Other C-native types like dict behave the same:

In [9]: class SubClass(dict): 
   ...:     def __new__(cls): 
   ...:         return super().__new__(cls) 
   ...:                                                                                                                  

In [10]: SubClass()                                                                                                      
Out[10]: {}

[davidhewitt]

Interesting. I have a feeling that C++'s pybind11 makes subclassing possible somehow, so I'll take some time to investigate this someday...

[birkenfeld]

@kngwyu I'm not sure the dict repr proves that the object isn't of type SubClass.

[kngwyu]

@birkenfeld
Ah, you're right, good catch.

In [6]: class SubClass(dict): 
   ...:     def __new__(cls): 
   ...:         return super().__new__(cls) 
   ...:          
   ...:                                                                                                                  

In [7]: c = SubClass()                                                                                                   

In [8]: c.__class__                                                                                                      
Out[8]: __main__.SubClass

[kngwyu]

I found a problem with our tp_new and am trying to fix it.