Convert DDS rosbag to iceoryx

[blackliner]

Hi, i have some rosbags which were recorded with the default ROS2 rmw (fast_RTPS) and i would like to do a performance comparison between different other rmw choices. Iceoryx sounds awesome, but i am not able to replay those rosbags. Is there something i missed? Do i need to build ROS2 together with iceoryx and rmw_iceoryx with some special env-variables set?
Cheers

[blackliner]

Ok, i am not even able to record anything via RMW_IMPLEMENTATION=rmw_iceoryx_cpp ros2 bag record -a -o iceoryx_test, is it possible that rosbag is not yet supported?

020-03-21 22:59:51.590 [ Info  ]: Application registered management segment 0x7f3c2e702000 with size 113371808 to id 1
2020-03-21 22:59:51.591 [ Info  ]: Application registered payload segment 0x7f3c0af53000 with size 595259200 to id 2
[INFO] [rosbag2_storage]: Opened database 'iceoryx_test/iceoryx_test_0.db3' for READ_WRITE.
2020-03-21 22:59:51.783 [Warning]: Cache size for subscribe too large 1000, limiting to MAX_RECEIVER_QUEUE_SIZE = 256
[INFO] [rosbag2_transport]: Listening for topics...
[INFO] [rosbag2_transport]: Subscribed to topic '/string'
[INFO] [rosbag2_transport]: Subscribed to topic '/rosout'
[ERROR] [rosbag2_transport]: Failed to record: package 'Introspection_ara_msgs' not found, searching: blablub

[Karsten1987]

yes, the rmw_iceoryx currently lacks a proper interface to serialize messages: https://github.com/ros2/rmw_iceoryx/blob/master/rmw_iceoryx_cpp/src/rmw_serialize.cpp

Thanks for opening this ticket though. I hope to get some time to get that serialization done in the next few days.

[budrus]

'Introspection_ara_msgs' are our built-in topics in iceoryx. For these we have no ROS messages. Does a bag record want to measure all the topics that are listed with ros2 topic list?
We could suppress the built-in topics (for a quick solution and later maybe have ROS messages for them)

Does this make sense?

[blackliner]

... or make them available as ROS2 messages? Or are they really, really special, so they can not be represented in the OMG IDL?
But i guess it is not really important to have them, so maybe hide them for now?

[budrus]

Exactly! They are needed mainly to get the iceoryx internals in the rmw. So we suppress them for now (commit is already on a branch)

ROS messages would be cool in future. Then you could have the information like memory pool usage also via ROS tooling. We will check if we can describe them with the ROS IDL. They use some of our special fixed size containers, so maybe this will not be straight forward

[Karsten1987]

@blackliner any chance you could give #15 a shot? With the latest rosbag2 master, you should be able to record rosbags with iceoryx.
I'll address the conversion in a separate PR.

[blackliner]

ROS2 eloquent

I tried all of that with the eloquent based package release from Ubuntu 18.04, i will try foxy from master later.

Of course.

✔️ Record by topic name

Recording with a named topic (in this case RMW_IMPLEMENTATION=rmw_iceoryx_cpp ros2 topic pub /bool_test std_msgs/msg/Bool "data: True") works:

bash output

fberchtold@FB-UBUNTU:~/drvline$ RMW_IMPLEMENTATION=rmw_iceoryx_cpp ros2 bag record /bool_test
2020-03-27 12:02:04.286 [ Debug ]: Application registered management segment 0x7ff86e321000 with size 201703392 to id 12020-03-27 12:02:04.287 [ Info  ]: Application registered payload segment 0x7ff84a934000 with size 595259200 to id 2
[INFO] [rosbag2_storage]: Opened database 'rosbag2_2020_03_27-12_02_04/rosbag2_2020_03_27-12_02_04_0.db3' for READ_WRITE.
2020-03-27 12:02:04.305 [Warning]: Cache size for subscribe too large 1000, limiting to MAX_RECEIVER_QUEUE_CAPACITY = 256
[INFO] [rosbag2_transport]: Listening for topics...
[INFO] [rosbag2_transport]: Subscribed to topic '/bool_test'
[INFO] [rosbag2_transport]: All requested topics are subscribed. Stopping discovery...
^C[INFO] [rclcpp]: signal_handler(signal_value=2)
fberchtold@FB-UBUNTU:~/drvline$ RMW_IMPLEMENTATION=rmw_iceoryx_cpp ros2 bag info

Files:             rosbag2_2020_03_27-12_02_04/rosbag2_2020_03_27-12_02_04_0.db3
Bag size:          16.5 KiB
Storage id:        sqlite3
Duration:          8.0s
Start:             Mar 27 2020 12:02:04.968 (1585306924.968)
End                Mar 27 2020 12:02:12.969 (1585306932.969)
Messages:          9
Topic information: Topic: /bool_test | Type: std_msgs/msg/Bool | Count: 9 | Serialization Format: shared_memory

❌ Record all topics via -a flag

Unfortunately, recording all topics via RMW_IMPLEMENTATION=rmw_iceoryx_cpp ros2 bag record -a results in:

bash output

2020-03-27 12:05:37.695 [ Debug ]: Application registered management segment 0x7fbe4eccb000 with size 201703392 to id 12020-03-27 12:05:37.696 [ Info  ]: Application registered payload segment 0x7fbe2b2de000 with size 595259200 to id 2
[INFO] [rosbag2_storage]: Opened database 'rosbag2_2020_03_27-12_05_37/rosbag2_2020_03_27-12_05_37_0.db3' for READ_WRITE.
2020-03-27 12:05:37.714 [Warning]: Cache size for subscribe too large 1000, limiting to MAX_RECEIVER_QUEUE_CAPACITY = 256
[INFO] [rosbag2_transport]: Listening for topics...
[ERROR] [rosbag2_transport]: Failed to record: package 'iceoryx_introspection_msgs' not found, searching: PATH

INFO: RMW_IMPLEMENTATION=rmw_iceoryx_cpp ros2 topic list is not able to find the /bool_test topic

[blackliner]

ROS2 foxy

✔️ Record all topics via -a flag

Now with ros2 foxy: Looks good, the record works now:

and:

❗ I don't know its my setup, but i get many many log messages when running any ros2 command ❗

bash output

fberchtold@FB-UBUNTU:~/ros2_foxy$ RMW_IMPLEMENTATION=rmw_iceoryx_cpp ros2 bag record -a

>>> [rcutils|error_handling.c:108] rcutils_set_error_state()
This error state is being overwritten:

  'failed to resolve symbol 'rmw_publisher_event_init' in shared library '/home/fberchtold/ros2_foxy/install/rmw_iceoryx_cpp/lib/librmw_iceoryx_cpp.so', at /home/fberchtold/ros2_foxy/src/ros2/rmw_implementation/rmw_implementation/src/functions.cpp:95'

with this new error message:

  'failed to resolve symbol 'rmw_subscription_event_init' in shared library '/home/fberchtold/ros2_foxy/install/rmw_iceoryx_cpp/lib/librmw_iceoryx_cpp.so', at /home/fberchtold/ros2_foxy/src/ros2/rmw_implementation/rmw_implementation/src/functions.cpp:95'

rcutils_reset_error() should be called after error handling to avoid this.
<<<

>>> [rcutils|error_handling.c:108] rcutils_set_error_state()
This error state is being overwritten:

  'failed to resolve symbol 'rmw_subscription_event_init' in shared library '/home/fberchtold/ros2_foxy/install/rmw_iceoryx_cpp/lib/librmw_iceoryx_cpp.so', at /home/fberchtold/ros2_foxy/src/ros2/rmw_implementation/rmw_implementation/src/functions.cpp:95'

with this new error message:

  'failed to resolve symbol 'rmw_get_publishers_info_by_topic' in shared library '/home/fberchtold/ros2_foxy/install/rmw_iceoryx_cpp/lib/librmw_iceoryx_cpp.so', at /home/fberchtold/ros2_foxy/src/ros2/rmw_implementation/rmw_implementation/src/functions.cpp:95'

rcutils_reset_error() should be called after error handling to avoid this.
<<<

>>> [rcutils|error_handling.c:108] rcutils_set_error_state()
This error state is being overwritten:

  'failed to resolve symbol 'rmw_get_publishers_info_by_topic' in shared library '/home/fberchtold/ros2_foxy/install/rmw_iceoryx_cpp/lib/librmw_iceoryx_cpp.so', at /home/fberchtold/ros2_foxy/src/ros2/rmw_implementation/rmw_implementation/src/functions.cpp:95'

with this new error message:

  'failed to resolve symbol 'rmw_get_subscriptions_info_by_topic' in shared library '/home/fberchtold/ros2_foxy/install/rmw_iceoryx_cpp/lib/librmw_iceoryx_cpp.so', at /home/fberchtold/ros2_foxy/src/ros2/rmw_implementation/rmw_implementation/src/functions.cpp:95'

rcutils_reset_error() should be called after error handling to avoid this.
<<<
2020-03-27 13:46:19.655 [ Debug ]: Application registered management segment 0x7ff85c059000 with size 201703392 to id 1
2020-03-27 13:46:19.655 [ Info  ]: Application registered payload segment 0x7ff83866c000 with size 595259200 to id 2
[INFO] [1585313179.675487411] [rosbag2_storage]: Opened database 'rosbag2_2020_03_27-13_46_19/rosbag2_2020_03_27-13_46_19_0.db3' for READ_WRITE.
2020-03-27 13:46:19.681 [Warning]: Cache size for subscribe too large 1000, limiting to MAX_RECEIVER_QUEUE_CAPACITY = 256
[INFO] [1585313179.681353532] [rosbag2_transport]: Listening for topics...
[WARN] [1585313179.732074110] [rosbag2_transport]: Hidden topics are not recorded. Enable them with --include-hidden-topics
[INFO] [1585313179.772584252] [rosbag2_transport]: Subscribed to topic '/rosout'
[INFO] [1585313179.777500870] [rosbag2_transport]: Subscribed to topic '/bool_test'
[INFO] [1585313179.778968275] [rosbag2_transport]: Subscribed to topic '/parameter_events'

[Karsten1987]

Thanks for the reports here. I think you assest the situation currently in that ros2 bag record -a is not support in for eloquent and you'd need to get a specific list of topics to be recorded.

As for the the missing functions, you've seen that #18 will add these and you shouldn't see these warnings any further.