resolving failed CI-job for eloquent

[JanStaschulat]

the rclc unit tests depend on these repositories (which are not in standard eloquent distribution):

• https://github.com/osrf/osrf_testing_tools_cpp.git
• https://github.com/ros2/rcl_interfaces.git which contains the package test_msgs

I updated the ci.yml file. Note, however to install the binary-package test_msgs of the repository rcl_interfaces, you need to:
apt-get install ros-DISTRO-test-msgs and not
apt-get install ros-DISTRO-rcl-interfaces!

---

Alternative: specify repositories in a file and clone them, like here:
https://github.com/boschresearch/fmi_adapter_ros2/blob/master/.github/workflows/build_and_test.yml
https://github.com/boschresearch/fmi_adapter_ros2/blob/master/dependencies.repos

However, this does not work for the dependencies of rclc repository:

1. osrf-testing-tools-cpp repository does not have a rolling branch.
2. The repo has branches for dashing, eloquent, foxy. So it would need a distinction between different branches. Or try out with master branch for all ROS 2 distributions.

---

Email history:

Von: Staschulat Jan

I found a difference in the CI job for Dashing and Eloquent in the command:

rosdep install

Dashing:

https://github.com/micro-ROS/rclc/runs/1686416342?check_suite_focus=true

The following NEW packages will be installed:
ros-dashing-osrf-testing-tools-cpp
0 upgraded, 1 newly installed, 0 to remove and 4 not upgraded.
Need to get 1,818 kB of archives.
After this operation, 2,127 kB of additional disk space will be used.
Get:1 http://packages.ros.org/ros2/ubuntu bionic/main amd64 ros-dashing-osrf-testing-tools-cpp amd64 1.2.3-1bionic.20201125.034205 [1,818 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 1,818 kB in 1s (1,938 kB/s)
Selecting previously unselected package ros-dashing-osrf-testing-tools-cpp.
(Reading database ...

Eloquent:

https://github.com/micro-ROS/rclc/runs/1686416315?check_suite_focus=true

Invoking "bash -c 'DEBIAN_FRONTEND=noninteractive RTI_NC_LICENSE_ACCEPTED=yes rosdep install -r --from-paths src --ignore-src --rosdistro eloquent -y || true'
/bin/bash -c DEBIAN_FRONTEND=noninteractive RTI_NC_LICENSE_ACCEPTED=yes rosdep install -r --from-paths src --ignore-src --rosdistro eloquent -y || true
WARNING: ROS_PYTHON_VERSION is unset. Defaulting to 2
ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
rclc_lifecycle: Cannot locate rosdep definition for [osrf_testing_tools_cpp]
rclc_examples: Cannot locate rosdep definition for [example_interfaces]
rclc: Cannot locate rosdep definition for [example_interfaces]
Continuing to install resolvable dependencies...
#All required rosdeps installed successfully
Invoking "bash -c 'source /opt/ros/eloquent/setup.sh && colcon build --event-handlers console_cohesion+ --packages-up-to rclc rclc_examples rclc_lifecycle --cmake-args --symlink-install'
Error: The process '/bin/bash' failed with exit code 1

Von: Lange Ralph

Hi Jan,

according to https://github.com/ros-tooling/action-ros-ci#build-with-a-custom-repos-or-rosinstall-file the necessary installation step can be automated without specifying the dependencies a third time.

However, last summer, for fmi_adapter, I failed with these instructions and finally added the dependency explicitly to a “dependencies.repos” file referenced in the Action workflow.

Mit freundlichen Grüßen / Best regards

Ralph Lange

​

---

In my local machine the package osrf_testing_tools_cpp is not part of /opt/ros/dashing, /opt/ros/eloquent ... and the other versions. So I had to 'git clone' the package osrf_testing_tools_cpp locally. When building rclc locally with "colcon bulid --packages-up-to rclc_examples" then the osrf_testing_tools are also build.

As a test, i removed the osrf_testing_tools_cpp package from my ros2 workspace, which contains the rclc package. building for Dashing AND Eloquent failed because that package was missing. This is interesting, because the 'colcon build' for Dashing works in the CI job at micro-ros server. Is the repository downloaded for the Dashing version?

The dependency for osrf_testing_tools_cpp is there in the package.xml as well as in CMakeLists.txt.

So solve the issue, we need to make sure that the workspace of the CI-job, in which the rclc repository is build, also downloads the package

https://github.com/osrf/osrf_testing_tools_cpp

How can this be done in the CI job specification?

---

Eloquent CI failing because something related with osrf_testing_tools:

https://github.com/micro-ROS/rclc/runs/1679637550?check_suite_focus=true

[JanStaschulat]

@pablogs9 @norro @ralph-lange this did not resolve the issue, we need to find a way to include the missing packages in the rosdep installation step.

Assuming 'source /opt/ros/eloquent/setup.bash'. I started with an empty workspace, cloned rclc repository and had do download these packages to build rclc for Eloquent:

- https://github.com/osrf/osrf_testing_tools_cpp.git
- https://github.com/ros2/rcl_interfaces.git
- https://github.com/ros2/test_interface_files.git

[pablogs9]

Why is not this package in eloquent distribution if it is listed here?
https://github.com/ros2/ros2/blob/eloquent-release/ros2.repos#L46

[JanStaschulat]

An alternative would be to refactor the unit tests and remove the dependancy of the osrf_testing_tools_cpp package.

[JanStaschulat]

@pablogs9 @ralph-lange

I tried to apt-get install osrf repo. Which works, but then test_msgs was not found, even though it was specified to install the corresponding package as well (and it was, by the way, not installed, because the most recent version was already there!) Very strange!

Then I tried with "git clone" all the dependencies, which also failed because the osrf_testing_tools_cpp could not be found. Probably the repo was cloned into some other directory, but not where
~/ws/src/rclc
is.
Does someone know into which directory the CI job puts the cloned rclc repo?

Some other question:
where do I find information about the syntax specification of the ci.yml file?

Specifically I'd like to know what the '|' is for:

• name: Download dependencies

run: |
git clone -b foxy https://github.com/osrf/osrf_testing_tools_cpp src/osrf_testing_tools_cpp

• name: Build
  run: . /opt/ros/foxy/setup.sh && colcon build
• name: Download dependencies

I googled a little bit but there is a lot of information about github ci yml stuff.
My question is: if I have multiple commands, do I need the | or can I just enter a command in every new line?

[JanStaschulat]

@ralph-lange @pablogs9

update:

With apt-get I could install osrf-testing-tools-cpp in /opt/ros/DISTRO in the ci.yml file configuration and the package is found in the build step.
However, now test_msgs is not found. The repository, which contains this package
https://github.com/ros2/rcl_interfaces
is installed in my pc. I setup locally on my machine:

sudo apt-get install ros-eloquent-osrf-testing-tools-cpp
sudo apt-get install ros-eloquent-rcl-interfaces
source /opt/ros/eloquent/setup.bash
mkdir -p ~/ros_ws/src
cd ~/ros_ws/src
git clone https://github.com/micro-ROS/rclc.git
cd ~/ros_ws; colcon_build --packages-up-to rclc

But here also the test_msgs could not be found. And indeed they are not part of /opt/ros/eloqent/share/rcl_interfaces/.

CMake Error at /opt/ros/eloquent/share/ament_cmake_target_dependencies/cmake/ament_target_dependencies.cmake:62 (message):
  ament_target_dependencies() the passed package name 'test_msgs' was not
  found before
Call Stack (most recent call first):
  CMakeLists.txt:117 (ament_target_dependencies)


-- Configuring incomplete, errors occurred!
See also "/home/jst3si/test_ws_ci/build/rclc/CMakeFiles/CMakeOutput.log".
See also "/home/jst3si/test_ws_ci/build/rclc/CMakeFiles/CMakeError.log".
--- stderr: rclc
CMake Error at /opt/ros/eloquent/share/ament_cmake_target_dependencies/cmake/ament_target_dependencies.cmake:62 (message):
  ament_target_dependencies() the passed package name 'test_msgs' was not
  found before

Do you have any idea, how to resolve this?
Is this intended? Do I have to clone and re-build such repos?

[JanStaschulat]

Found the same issue here:
ros2/rosbag2#97

The correct name for test_msgs package is:

apt-get install ros-${{ matrix.ros_distribution }}-test-msgs

[JanStaschulat]

now it builds again :-)

please approve.

[norro]

cudos ;)