Add python3-dev build_depend to lttngpy.

[clalancette]

This is needed because the CMakeLists.txt file for this package does:

find_package(Python3 REQUIRED COMPONENTS Interpreter Development)

Otherwise, it may fail to build on certain platforms on the buildfarm (like RHEL-9).

I believe this will fix the failing build in https://build.ros2.org/view/Rbin_rhel_el964/job/Rbin_rhel_el964__lttngpy__rhel_9_x86_64__binary/

[christophebedard]

Seems reasonable to me. However, I'm seeing 7 instances of

find_package(Python3 REQUIRED COMPONENTS Interpreter Development)

in CMakeLists.txt files but no instances of

<build_depend>python3-dev</build_depend>

python_cmake_module is the only package with any kind of dependency (buildtool_export_depend) on python3-dev. Should more packages also use this? For example, rclpy.

[clalancette]

python_cmake_module is the only package with any kind of dependency (buildtool_export_depend) on python3-dev. Should more packages also use this? For example, rclpy.

Hm, that is a really good point. I wonder why rclpy is not failing to build on RHEL for similar reasons. I'll take a closer look to see what I can figure out.

[christophebedard]

Maybe there are indirect dependencies, and potentially other packages (not python3-dev) providing whatever lttngpy needs here?

[clalancette]

Maybe there are indirect dependencies, and potentially other packages (not python3-dev) providing whatever lttngpy needs here?

Well, lttngpy is definitely failing to build on RHEL-9; that's clear from the job on the buildfarm. And it directly depends on python3-dev, so we need it here.

But it does look like another dependency pulls in python3-dev for rclpy: https://build.ros2.org/job/Rbin_rhel_el964__rclpy__rhel_9_x86_64__binary/116/consoleFull. I'm trying to figure out which one it is. Regardless, we probably need PRs to some of those other packages to directly depend on python3-dev as well.

I'm also trying to figure out whether it should be build_depend or buildtool_export_depend (or both).

[clalancette]

Oh boy, this is a bit of a rabbit-hole.

It is indeed the case that this package is missing a dependency, which is why it is failing to build on the buildfarm. It is also the case that several other packages are missing dependencies, but it ends up not mattering for frustrating reasons.

First, the CMake documentation for pybind11 explicitly states that you should call find_package(Python3 REQUIRED COMPONENTS Interpreter Development) before doing find_package(pybind11 REQUIRED): https://github.com/pybind/pybind11/blob/83b92ceb3537666fb0188f564e1d53bf8c80b0ba/tools/pybind11Config.cmake.in#L73 . So we are doing the correct thing there in all of these packages. In turn, that means that all of our packages that depend on pybind11 should have a dependency on python3-dev. In reality, none of them have that dependency specified.

However, in all cases but this package, it ends up not mattering, because they get the dependency indirectly. It turns out that the rosidl pipeline specifies a dependency on python3: https://github.com/ros2/rosidl/blob/2a808dc7b449d43f80ce22bc2fe3b8e31c96ca3b/rosidl_generator_c/package.xml#L24 . In theory, having a dependency on python3 wouldn't be enough, but it turns out that (probably for historical reasons) the rosdep key for python3 also includes the development libraries: https://github.com/ros/rosdistro/blob/ffa9b38dbb20d7a89bc3d99254daf91b9d0083d0/rosdep/python.yaml#L4727-L4738 . Because all of the other packages depend on some part of the rosidl pipeline (either directly or indirectly), they all get the dependency installed and thus all compile.

Because of the above, the packages that depend on pybind11 really should have a dependency on python3-dev. The last question to answer here is what kind of dependency. Clearly it has to be one of the build.*_depend, because it is needed at build time. pybind11 is a header-only library, and it has no external dependencies other than Eigen (which is also header-only). but what is compiled at the end of the day is a .so for the target machine. Looking at https://www.ros.org/reps/rep-0140.html , I think that the most appropriate dependency here would be build_depend, since it is needed during compilation on the target, but doesn't need to be exported downstream.

Thus, my suggestion is that this PR is correct, and that we should do a similar PR for rclpy, rosbag2_py, image_transport_py, point_cloud_transport_py, tf2_py, tf2_sensor_msgs, tf2_geometry_msgs, and qt_gui_cpp (all of the other core packages that call find_package(Python3 REQUIRED COMPONENTS Interpreter Development)).

@christophebedard @sloretz Can you double-check my work here, and see if everything I laid out above makes sense? If so, I'll go ahead and make all of these PRs tomorrow so we can try to get them in to do another round of Rolling releases.

[christophebedard]

Wow! Thanks for the investigation. That all makes sense.

[clalancette]

Pulls: #146
Gist: https://gist.githubusercontent.com/clalancette/e882d728ec6676e9b40f33f70b3853b3/raw/3e9bd255a8ab5b2bb47a4ed951c05f70479ab9e6/ros2.repos
BUILD args: --packages-above-and-dependencies lttngpy
TEST args: --packages-above lttngpy
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/14850

• Linux
• Linux-aarch64
• Linux-rhel
• Windows