issue with cython headers for external projects

[mccullerlp]

The current version of pyzmq gives the .pxd files needed for cython to expose zmq objects, but all of the high level objects in zmq/core directly or indirectly link to libzmq.pxd. The problem with this is that libzmq references some headers that no python/cython program should need and aren't included with many packages.

Making C extension libraries that can use pyzmq sockets for instance can't currently cimport zmq.core.socket because socket cimports message (unnecessarily) and message cimports libzmq.

It would be helpful to remove unecessary references to libzmq where possible, or to split libzmq ether apart, to prevent references to implementation headers, or to give purpose-built user-facing .pxd files that can be cimported without these problems.

[minrk]

I'll remove the message cimport from socket.pxd, thanks. I believe that was lingering from an old API when Message was referenced in one of the Socket cdef methods. I have no experience building libraries that should be cimportable, so any help/guidance is useful, though I know gevent-zeromq has had success. Does removing that one cimport solve your issue?

libzmq.pxd is simply a cython expression of zmq.h, so I don't see how it would be useful if you don't have zmq.h. Can you describe the exact issue there?

For instance, how would you express zmq_msg_t in message.pxd without depending on zmq.h, or am I misunderstanding the issue?

[mccullerlp]

For sockets, removing cimport message resolves the issue perfectly.

For message.pxd and anything else depending on libzmq, the problem is that
when you try to compile the .c file of your external library, you get
unresolved references to "allocate.h" and "zmq_compat.h", which look to be
part of the pyzmq internal implementation. I think that if those references
were moved to a separate .pxd file and only referenced from the libzmq .pyx
implementation, the problem would be resolved for anything referencing
libzmq. References to zmq.h should be fine since any user will have it
installed.

Thanks for the fixes! This should make creating the implementation of fast
routing devices in c/cython extremely easy. I'm considering writing a quick
guide on how to do it.

On Sat, Feb 18, 2012 at 1:46 PM, Min RK <
[email protected]

wrote:

I'll remove the message cimport from socket.pxd, thanks. I believe that
was lingering from an old API when Message was referenced in one of the
Socket cdef methods. I have no experience building libraries that should
be cimportable, so any help/guidance is useful, though I know
gevent-zeromq has had
success. Does removing that one cimport solve your issue?

libzmq.pxd is simply a cython expression of zmq.h, so I don't see how it
would be useful if you don't have zmq.h. Can you describe the exact issue
there?

For instance, how would you express zmq_msg_t in message.pxd without
depending on zmq.h, or am I misunderstanding the issue?

---

Reply to this email directly or view it on GitHub:
#178 (comment)

[minrk]

allocate.h and zmq_compat.h ship with pyzmq in zmq/utils, so you should always have them if you have pyzmq. I did just check, and the path reported by zmq.get_includes() for zmq.utils was wrong, so that too will be fixed when PR #172 is merged. I will think about splitting off allocate.h and some of the generic utilities (memcpy, ec.) from libzmq.pxd, but I cannot do that with zmq_compat.h, which is required for libzmq.pxd to work with zmq.h from both libzmq-2.1 and 3.1.