WIP: PyPy support

.travis.yml

@@ -29,6 +29,13 @@ matrix:
   - os: osx
     osx_image: xcode7.3
     env: PYTHON=3.5 CPP=14 CLANG
+  # Test a PyPy 2.7 nightly build
+  - os: linux
+    env: PYPY=1 PYTHON=2.7 CPP=11 GCC=4.8
+    addons:
+      apt:
+        sources: [ubuntu-toolchain-r-test, kubuntu-backports]
+        packages: [g++-4.8, cmake]
   # A barebones build makes sure everything still works without optional deps (numpy/scipy/eigen)
   # and also tests the automatic discovery functions in CMake (Python version, C++ standard).
   - os: linux
@@ -70,9 +77,18 @@ before_install:
   fi
   if [ -n "$CPP" ]; then export CPP=-std=c++$CPP; fi
   if [ "${PYTHON:0:1}" = "3" ]; then export PY=3; fi
+  if [ -n "$PYPY" ]; then
+    curl http://buildbot.pypy.org/nightly/trunk/pypy-c-jit-latest-linux64.tar.bz2 | tar -xj
+    export PYPY_BINARY=$(echo `pwd`/pypy-c-jit*/bin/pypy)
+    export CMAKE_EXTRA_ARGS="-DPYTHON_EXECUTABLE:FILEPATH=$PYPY_BINARY"
+  fi
   if [ -n "$DEBUG" ]; then export CMAKE_EXTRA_ARGS="-DCMAKE_BUILD_TYPE=Debug"; fi
 - |
-  # Initialize enviornment
+  # Initialize environment
+  if [ -n "$PYPY" ]; then
+    $PYPY_BINARY -m ensurepip
+    $PYPY_BINARY -m pip install pytest
+  fi
   if [ -n "$DOCKER" ]; then
     docker pull $DOCKER
     export containerid=$(docker run --detach --tty \

README.md

@@ -25,7 +25,7 @@ become an excessively large and unnecessary dependency.
 Think of this library as a tiny self-contained version of Boost.Python with
 everything stripped away that isn't relevant for binding generation. Without
 comments, the core header files only require ~2.5K lines of code and depend on
-Python (2.7 or 3.x) and the C++ standard library. This compact implementation
+Python (2.7 or 3.x, or PyPy2.7 >= 5.5) and the C++ standard library. This compact implementation
 was possible thanks to some of the new C++11 language features (specifically:
 tuples, lambda functions and variadic templates). Since its creation, this
 library has grown beyond Boost.Python in many ways, leading to dramatically
@@ -58,12 +58,15 @@ pybind11 can map the following core C++ features to Python
 ## Goodies
 In addition to the core functionality, pybind11 provides some extra goodies:
 
-- pybind11 uses C++11 move constructors and move assignment operators whenever
-  possible to efficiently transfer custom data types.
+- Python 2.7, 3.x, and PyPy (PyPy2.7 >= 5.5) are supported with an
+  implementation-agnostic interface.
 
 - It is possible to bind C++11 lambda functions with captured variables. The
   lambda capture data is stored inside the resulting Python function object.
 
+- pybind11 uses C++11 move constructors and move assignment operators whenever
+  possible to efficiently transfer custom data types.
+
 - It's easy to expose the internal storage of custom data types through
   Pythons' buffer protocols. This is handy e.g. for fast conversion between
   C++ matrix classes like Eigen and NumPy without expensive copy operations.
@@ -100,7 +103,7 @@ In addition to the core functionality, pybind11 provides some extra goodies:
 
 ## About
 
-This project was created by [Wenzel Jakob](https://www.mitsuba-renderer.org/~wenzel/).
+This project was created by [Wenzel Jakob](http://rgl.epfl.ch/people/wjakob).
 Significant features and/or improvements to the code were contributed by
 Jonas Adler,
 Sylvain Corlay,

docs/advanced/classes.rst

@@ -422,15 +422,24 @@ The section on :ref:`properties` discussed the creation of instance properties
 that are implemented in terms of C++ getters and setters.
 
 Static properties can also be created in a similar way to expose getters and
-setters of static class attributes. It is important to note that the implicit
-``self`` argument also exists in this case and is used to pass the Python
-``type`` subclass instance. This parameter will often not be needed by the C++
-side, and the following example illustrates how to instantiate a lambda getter
-function that ignores it:
+setters of static class attributes. Two things are important to note:
+
+1. Static properties are implemented by instrumenting the *metaclass* of the
+   class in question -- however, this requires the class to have a modifiable
+   metaclass in the first place. pybind11 provides a ``py::metaclass()``
+   annotation that must be specified in the ``class_`` constructor, or any
+   later method calls to ``def_{property_,∅}_{readwrite,readonly}_static`` will
+   fail (see the example below).
+
+2. For static properties defined in terms of setter and getter functions, note
+   that the implicit ``self`` argument also exists in this case and is used to
+   pass the Python ``type`` subclass instance. This parameter will often not be
+   needed by the C++ side, and the following example illustrates how to
+   instantiate a lambda getter function that ignores it:
 
 .. code-block:: cpp
 
-    py::class_<Foo>(m, "Foo")
+    py::class_<Foo>(m, "Foo", py::metaclass())
         .def_property_readonly_static("foo", [](py::object /* self */) { return Foo(); });
 
 Operator overloading

docs/advanced/pycpp/numpy.rst

@@ -33,7 +33,7 @@ completely avoid copy operations with Python expressions like
 
 .. code-block:: cpp
 
-    py::class_<Matrix>(m, "Matrix")
+    py::class_<Matrix>(m, "Matrix", py::buffer_protocol())
        .def_buffer([](Matrix &m) -> py::buffer_info {
             return py::buffer_info(
                 m.data(),                               /* Pointer to buffer */
@@ -46,9 +46,12 @@ completely avoid copy operations with Python expressions like
             );
         });
 
-The snippet above binds a lambda function, which can create ``py::buffer_info``
-description records on demand describing a given matrix. The contents of
-``py::buffer_info`` mirror the Python buffer protocol specification.
+Supporting the buffer protocol in a new type involves specifying the special
+``py::buffer_protocol()`` tag in the ``py::class_`` constructor and calling the
+``def_buffer()`` method with a lambda function that creates a
+``py::buffer_info`` description record on demand describing a given matrix
+instance. The contents of ``py::buffer_info`` mirror the Python buffer protocol
+specification.
 
 .. code-block:: cpp
 
@@ -77,7 +80,7 @@ buffer objects (e.g. a NumPy matrix).
     typedef Matrix::Scalar Scalar;
     constexpr bool rowMajor = Matrix::Flags & Eigen::RowMajorBit;
 
-    py::class_<Matrix>(m, "Matrix")
+    py::class_<Matrix>(m, "Matrix", py::buffer_protocol())
         .def("__init__", [](Matrix &m, py::buffer b) {
             typedef Eigen::Stride<Eigen::Dynamic, Eigen::Dynamic> Strides;
 

docs/intro.rst

@@ -51,6 +51,9 @@ Goodies
 *******
 In addition to the core functionality, pybind11 provides some extra goodies:
 
+- Python 2.7, 3.x, and PyPy (PyPy2.7 >= 5.5) are supported with an
+  implementation-agnostic interface.
+
 - It is possible to bind C++11 lambda functions with captured variables. The
   lambda capture data is stored inside the resulting Python function object.
 

include/pybind11/attr.h

@@ -47,6 +47,12 @@ struct multiple_inheritance { };
 /// Annotation which enables dynamic attributes, i.e. adds `__dict__` to a class
 struct dynamic_attr { };
 
+/// Annotation which enables the buffer protocol for a type
+struct buffer_protocol { };
+
+/// Annotation which requests that a special metaclass is created for a type
+struct metaclass { };
+
 /// Annotation to mark enums as an arithmetic type
 struct arithmetic { };
 
@@ -136,7 +142,9 @@ struct function_record {
 
 /// Special data structure which (temporarily) holds metadata about a bound class
 struct type_record {
-    PYBIND11_NOINLINE type_record() { }
+    PYBIND11_NOINLINE type_record()
+        : multiple_inheritance(false), dynamic_attr(false),
+          buffer_protocol(false), metaclass(false) { }
 
     /// Handle to the parent scope
     handle scope;
@@ -166,10 +174,16 @@ struct type_record {
     const char *doc = nullptr;
 
     /// Multiple inheritance marker
-    bool multiple_inheritance = false;
+    bool multiple_inheritance : 1;
 
     /// Does the class manage a __dict__?
-    bool dynamic_attr = false;
+    bool dynamic_attr : 1;
+
+    /// Does the class implement the buffer protocol?
+    bool buffer_protocol : 1;
+
+    /// Does the class require its own metaclass?
+    bool metaclass : 1;
 
     PYBIND11_NOINLINE void add_base(const std::type_info *base, void *(*caster)(void *)) {
         auto base_info = detail::get_type_info(*base, false);
@@ -309,6 +323,16 @@ struct process_attribute<dynamic_attr> : process_attribute_default<dynamic_attr>
     static void init(const dynamic_attr &, type_record *r) { r->dynamic_attr = true; }
 };
 
+template <>
+struct process_attribute<buffer_protocol> : process_attribute_default<buffer_protocol> {
+    static void init(const buffer_protocol &, type_record *r) { r->buffer_protocol = true; }
+};
+
+template <>
+struct process_attribute<metaclass> : process_attribute_default<metaclass> {
+    static void init(const metaclass &, type_record *r) { r->metaclass = true; }
+};
+
 
 /// Process an 'arithmetic' attribute for enums (does nothing here)
 template <>

include/pybind11/cast.h

@@ -108,14 +108,10 @@ PYBIND11_NOINLINE inline handle get_type_handle(const std::type_info &tp, bool t
 }
 
 PYBIND11_NOINLINE inline bool isinstance_generic(handle obj, const std::type_info &tp) {
-    const auto type = detail::get_type_handle(tp, false);
+    handle type = detail::get_type_handle(tp, false);
     if (!type)
         return false;
-
-    const auto result = PyObject_IsInstance(obj.ptr(), type.ptr());
-    if (result == -1)
-        throw error_already_set();
-    return result != 0;
+    return isinstance(obj, type);
 }
 
 PYBIND11_NOINLINE inline std::string error_string() {
@@ -141,6 +137,7 @@ PYBIND11_NOINLINE inline std::string error_string() {
         PyException_SetTraceback(scope.value, scope.trace);
 #endif
 
+#if !defined(PYPY_VERSION)
     if (scope.trace) {
         PyTracebackObject *trace = (PyTracebackObject *) scope.trace;
 
@@ -160,6 +157,7 @@ PYBIND11_NOINLINE inline std::string error_string() {
         }
         trace = trace->tb_next;
     }
+#endif
 
     return errorString;
 }
@@ -176,7 +174,9 @@ PYBIND11_NOINLINE inline handle get_object_handle(const void *ptr, const detail:
 }
 
 inline PyThreadState *get_thread_state_unchecked() {
-#if   PY_VERSION_HEX < 0x03000000
+#if defined(PYPY_VERSION)
+    return PyThreadState_GET();
+#elif PY_VERSION_HEX < 0x03000000
     return _PyThreadState_Current;
 #elif PY_VERSION_HEX < 0x03050000
     return (PyThreadState*) _Py_atomic_load_relaxed(&_PyThreadState_Current);
@@ -224,7 +224,7 @@ class type_caster_generic {
 
             /* If this is a python class, also check the parents recursively */
             auto const &type_dict = get_internals().registered_types_py;
-            bool new_style_class = PyType_Check(tobj);
+            bool new_style_class = PyType_Check((PyObject *) tobj);
             if (type_dict.find(tobj) == type_dict.end() && new_style_class && tobj->tp_bases) {
                 auto parents = reinterpret_borrow<tuple>(tobj->tp_bases);
                 for (handle parent : parents) {
@@ -662,10 +662,10 @@ template <> class type_caster<std::wstring> {
 #if PY_MAJOR_VERSION >= 3
         buffer = PyUnicode_AsWideCharString(load_src.ptr(), &length);
 #else
-        temp = reinterpret_steal<object>(
-            sizeof(wchar_t) == sizeof(short)
-                ? PyUnicode_AsUTF16String(load_src.ptr())
-                : PyUnicode_AsUTF32String(load_src.ptr()));
+        temp = reinterpret_steal<object>(PyUnicode_AsEncodedString(
+            load_src.ptr(), sizeof(wchar_t) == sizeof(short)
+            ? "utf16" : "utf32", nullptr));
+
         if (temp) {
             int err = PYBIND11_BYTES_AS_STRING_AND_SIZE(temp.ptr(), (char **) &buffer, &length);
             if (err == -1) { buffer = nullptr; }  // TypeError
@@ -868,7 +868,7 @@ template <typename type, typename holder_type> class type_caster_holder : public
 
             /* If this is a python class, also check the parents recursively */
             auto const &type_dict = get_internals().registered_types_py;
-            bool new_style_class = PyType_Check(tobj);
+            bool new_style_class = PyType_Check((PyObject *) tobj);
             if (type_dict.find(tobj) == type_dict.end() && new_style_class && tobj->tp_bases) {
                 auto parents = reinterpret_borrow<tuple>(tobj->tp_bases);
                 for (handle parent : parents) {

include/pybind11/eval.h

@@ -95,8 +95,15 @@ object eval_file(str fname, object global = object(), object local = object()) {
         pybind11_fail("File \"" + fname_str + "\" could not be opened!");
     }
 
+#if PY_VERSION_HEX < 0x03000000 && defined(PYPY_VERSION)
+    PyObject *result = PyRun_File(f, fname_str.c_str(), start, global.ptr(),
+                                  local.ptr());
+    (void) closeFile;
+#else
     PyObject *result = PyRun_FileEx(f, fname_str.c_str(), start, global.ptr(),
                                     local.ptr(), closeFile);
+#endif
+
     if (!result)
         throw error_already_set();
     return reinterpret_steal<object>(result);

include/pybind11/functional.h

@@ -39,7 +39,7 @@ struct type_caster<std::function<Return(Args...) PYBIND11_NOEXCEPT_SPECIFIER>> {
            captured variables), in which case the roundtrip can be avoided.
          */
         if (PyCFunction_Check(src_.ptr())) {
-            auto c = reinterpret_borrow<capsule>(PyCFunction_GetSelf(src_.ptr()));
+            auto c = reinterpret_borrow<capsule>(PyCFunction_GET_SELF(src_.ptr()));
             auto rec = (function_record *) c;
 
             if (rec && rec->is_stateless && rec->data[1] == &typeid(function_type)) {