Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

multiple camera issue #2814

Closed
ghost opened this issue Jul 18, 2023 · 7 comments
Closed

multiple camera issue #2814

ghost opened this issue Jul 18, 2023 · 7 comments
Labels
D400 l500

Comments

@ghost
Copy link

ghost commented Jul 18, 2023

ASAP 💨

Whenever I turn on the 3 cameras with the launch file below, the computer freezes, or
only 2 of the 3 cameras publish rostopic, and the other one doesn't publish topic even
though the rosnode is alive. Can I get some help ASAP?

For reference, the hardware spec is RTX3080 on an i9 10th Gen.

<launch>
  <arg name="serial_no_camera1"         default=""/> 			<!-- Note: D435i -->
  <arg name="serial_no_camera2"    	default=""/> 			<!-- Note: D415 -->
  <arg name="serial_no_camera3"         default=""/>      <!-- Note: L515 -->
  <arg name="camera1"              			default="camera"/>		<!-- Note: Replace with camera name -->
  <arg name="camera2"              			default="camera01"/>		<!-- Note: Replace with camera name -->
  <arg name="camera3"                   default="camera02"/>
  <arg name="tf_prefix_camera1"         default="$(arg camera1)"/>
  <arg name="tf_prefix_camera2"         default="$(arg camera2)"/>
  <arg name="tf_prefix_camera3"         default="$(arg camera3)"/>
  <arg name="initial_reset"             default="true"/>

  <arg name="external_manager"    default="false"/>
  <arg name="manager"             default="realsense2_camera_manager"/>

  <!-- Camera device specific arguments -->

  <arg name="usb_port_id"         default=""/>
  <arg name="device_type"         default=""/>
  <arg name="json_file_path"      default=""/>

  <arg name="depth_width"         default="640"/>
  <arg name="depth_height"        default="480"/>
  <arg name="enable_depth"        default="true"/>

  <arg name="fisheye_width"       default="640"/>
  <arg name="fisheye_height"      default="480"/>
  <arg name="enable_fisheye"      default="false"/>

  <arg name="infra_width"         default="640"/>
  <arg name="infra_height"        default="480"/>
  <arg name="enable_infra"       default="true"/>
  <arg name="enable_infra1"       default="true"/>
  <arg name="enable_infra2"       default="true"/>

  <arg name="color_width"         default="640"/>
  <arg name="color_height"        default="480"/>
  <arg name="enable_color"        default="true"/>

  <arg name="fisheye_fps"         default="30"/>
  <arg name="depth_fps"           default="30"/>
  <arg name="infra_fps"           default="30"/>
  <arg name="color_fps"           default="30"/>
  <arg name="gyro_fps"            default="400"/>
  <arg name="accel_fps"           default="400"/>
  <arg name="enable_gyro"         default="false"/>
  <arg name="enable_accel"        default="false"/>

  <arg name="enable_pointcloud"   default="true"/>
  <arg name="enable_sync"         default="true"/>
  <arg name="align_depth"         default="true"/>
  <arg name="filters"             default="temporal"/>
  <arg name="unite_imu_method"    default=""/>

  <!-- rgbd_launch specific arguments -->

  <!-- Arguments for remapping all device namespaces -->
  <arg name="rgb"                             default="color" />
  <arg name="ir"                              default="infra" />
  <arg name="depth"                           default="depth" />
  <arg name="depth_registered_pub"            default="depth_registered" />
  <arg name="depth_registered"                default="depth_registered" unless="$(arg align_depth)" />
  <arg name="depth_registered"                default="aligned_depth_to_color" if="$(arg align_depth)" />
  <arg name="depth_registered_filtered"       default="$(arg depth_registered)" />
  <arg name="projector"                       default="projector" />

  <!-- Disable bond topics by default -->
  <arg name="bond"                            default="false" />
  <arg name="respawn"                         default="$(arg bond)" />

  <!-- Processing Modules -->
  <arg name="rgb_processing"                  default="true"/>
  <arg name="debayer_processing"              default="false" />
  <arg name="ir_processing"                   default="false"/>
  <arg name="depth_processing"                default="false"/>
  <arg name="depth_registered_processing"     default="true"/>
  <arg name="disparity_processing"            default="false"/>
  <arg name="disparity_registered_processing" default="false"/>
  <arg name="hw_registered_processing"        default="$(arg align_depth)" />
  <arg name="sw_registered_processing"        default="true" unless="$(arg align_depth)" />
  <arg name="sw_registered_processing"        default="false" if="$(arg align_depth)" />

  <group ns="$(arg camera1)">

    <!-- Launch the camera device nodelet-->
    <include file="$(find realsense2_camera)/launch/includes/nodelet.launch.xml">
      <arg name="external_manager"         value="$(arg external_manager)"/>
      <arg name="manager"                  value="$(arg manager)"/>
      <arg name="tf_prefix"                value="$(arg tf_prefix_camera1)"/>
      <arg name="serial_no"                value="$(arg serial_no_camera1)"/>
      <arg name="usb_port_id"              value="$(arg usb_port_id)"/>
      <arg name="device_type"              value="$(arg device_type)"/>
      <arg name="json_file_path"           value="$(arg json_file_path)"/>

      <arg name="enable_pointcloud"        value="$(arg enable_pointcloud)"/>
      <arg name="enable_sync"              value="$(arg enable_sync)"/>
      <arg name="align_depth"              value="$(arg align_depth)"/>

      <arg name="fisheye_width"            value="$(arg fisheye_width)"/>
      <arg name="fisheye_height"           value="$(arg fisheye_height)"/>
      <arg name="enable_fisheye"           value="$(arg enable_fisheye)"/>
      <arg name="enable_depth"             value="$(arg enable_depth)"/>
      <arg name="enable_color"             value="$(arg enable_color)"/>
      
      <arg name="depth_width"       value="640"/>
      <arg name="depth_height"      value="480"/>
      <arg name="depth_fps"         value="30"/>
      <arg name="color_width"       value="640"/>
      <arg name="color_height"      value="480"/>
      <arg name="color_fps"         value="30"/>

      <arg name="infra_width"              value="$(arg infra_width)"/>
      <arg name="infra_height"             value="$(arg infra_height)"/>
      <arg name="enable_infra1"            value="$(arg enable_infra1)"/>
      <arg name="enable_infra2"            value="$(arg enable_infra2)"/>

      <arg name="fisheye_fps"              value="$(arg fisheye_fps)"/>
      <arg name="infra_fps"                value="$(arg infra_fps)"/>

      <arg name="gyro_fps"                 value="$(arg gyro_fps)"/>
      <arg name="accel_fps"                value="$(arg accel_fps)"/>
      <arg name="enable_gyro"              value="$(arg enable_gyro)"/>
      <arg name="enable_accel"             value="$(arg enable_accel)"/>
      <arg name="filters"                  value="$(arg filters)"/>
    </include>

    <!-- RGB processing -->
    <include if="$(arg rgb_processing)"
             file="$(find rgbd_launch)/launch/includes/rgb.launch.xml">
      <arg name="manager"                       value="$(arg manager)" />
      <arg name="respawn"                       value="$(arg respawn)" />
      <arg name="rgb"                           value="$(arg rgb)" />
      <arg name="debayer_processing"            value="$(arg debayer_processing)" />
    </include>

    <group if="$(eval depth_registered_processing and sw_registered_processing)">
      <node pkg="nodelet" type="nodelet" name="register_depth"
            args="load depth_image_proc/register $(arg manager) $(arg bond)" respawn="$(arg respawn)">
        <remap from="rgb/camera_info"             to="$(arg rgb)/camera_info" />
        <remap from="depth/camera_info"           to="$(arg depth)/camera_info" />
        <remap from="depth/image_rect"            to="$(arg depth)/image_rect_raw" />
        <remap from="depth_registered/image_rect" to="$(arg depth_registered)/sw_registered/image_rect_raw" />
      </node>

      <!-- Publish registered XYZRGB point cloud with software registered input -->
      <node pkg="nodelet" type="nodelet" name="points_xyzrgb_sw_registered"
            args="load depth_image_proc/point_cloud_xyzrgb $(arg manager) $(arg bond)" respawn="$(arg respawn)">
        <remap from="rgb/image_rect_color"        to="$(arg rgb)/image_rect_color" />
        <remap from="rgb/camera_info"             to="$(arg rgb)/camera_info" />
        <remap from="depth_registered/image_rect" to="$(arg depth_registered_filtered)/sw_registered/image_rect_raw" />
        <remap from="depth_registered/points"     to="$(arg depth_registered)/points" />
      </node>
    </group>

    <group if="$(eval depth_registered_processing and hw_registered_processing)">
      <!-- Publish registered XYZRGB point cloud with hardware registered input (ROS Realsense depth alignment) -->
      <node pkg="nodelet" type="nodelet" name="points_xyzrgb_hw_registered"
            args="load depth_image_proc/point_cloud_xyzrgb $(arg manager) $(arg bond)" respawn="$(arg respawn)">
        <remap from="rgb/image_rect_color"        to="$(arg rgb)/image_rect_color" />
        <remap from="rgb/camera_info"             to="$(arg rgb)/camera_info" />
        <remap from="depth_registered/image_rect" to="$(arg depth_registered)/image_raw" />
        <remap from="depth_registered/points"     to="$(arg depth_registered_pub)/points" />
      </node>
    </group>

  </group>

  <group ns="$(arg camera2)">
    <!-- Launch the camera device nodelet-->
    <include file="$(find realsense2_camera)/launch/includes/nodelet.launch.xml">
      <arg name="external_manager"         value="$(arg external_manager)"/>
      <arg name="manager"                  value="$(arg manager)"/>
      <arg name="tf_prefix"                value="$(arg tf_prefix_camera2)"/>
      <arg name="serial_no"                value="$(arg serial_no_camera2)"/>
      <arg name="usb_port_id"              value="$(arg usb_port_id)"/>
      <arg name="device_type"              value="$(arg device_type)"/>
      <arg name="json_file_path"           value="$(arg json_file_path)"/>

      <arg name="enable_pointcloud"        value="$(arg enable_pointcloud)"/>
      <arg name="enable_sync"              value="$(arg enable_sync)"/>
      <arg name="align_depth"              value="$(arg align_depth)"/>

      <arg name="depth_width"              value="$(arg depth_width)"/>
      <arg name="depth_height"             value="$(arg depth_height)"/>
      <arg name="enable_depth"             value="$(arg enable_depth)"/>

      <arg name="infra_width"              value="$(arg infra_width)"/>
      <arg name="infra_height"             value="$(arg infra_height)"/>
      <arg name="enable_infra"            value="$(arg enable_infra)"/>

      <arg name="color_width"              value="$(arg color_width)"/>
      <arg name="color_height"             value="$(arg color_height)"/>
      <arg name="enable_color"             value="$(arg enable_color)"/>

      <arg name="depth_fps"                value="$(arg depth_fps)"/>
      <arg name="infra_fps"                value="$(arg infra_fps)"/>
      <arg name="color_fps"                value="$(arg color_fps)"/>
      <arg name="gyro_fps"                 value="$(arg gyro_fps)"/>
      <arg name="accel_fps"                value="$(arg accel_fps)"/>
      <arg name="enable_gyro"              value="$(arg enable_gyro)"/>
      <arg name="enable_accel"             value="$(arg enable_accel)"/>
      <arg name="filters"                  value="$(arg filters)"/>
      <arg name="unite_imu_method"         value="$(arg unite_imu_method)"/>
    </include>

    <!-- RGB processing -->
    <include if="$(arg rgb_processing)"
             file="$(find rgbd_launch)/launch/includes/rgb.launch.xml">
      <arg name="manager"                       value="$(arg manager)" />
      <arg name="respawn"                       value="$(arg respawn)" />
      <arg name="rgb"                           value="$(arg rgb)" />
      <arg name="debayer_processing"            value="$(arg debayer_processing)" />
    </include>

    <group if="$(eval depth_registered_processing and sw_registered_processing)">
      <node pkg="nodelet" type="nodelet" name="register_depth"
            args="load depth_image_proc/register $(arg manager) $(arg bond)" respawn="$(arg respawn)">
        <remap from="rgb/camera_info"             to="$(arg rgb)/camera_info" />
        <remap from="depth/camera_info"           to="$(arg depth)/camera_info" />
        <remap from="depth/image_rect"            to="$(arg depth)/image_rect_raw" />
        <remap from="depth_registered/image_rect" to="$(arg depth_registered)/sw_registered/image_rect_raw" />
      </node>

      <!-- Publish registered XYZRGB point cloud with software registered input -->
      <node pkg="nodelet" type="nodelet" name="points_xyzrgb_sw_registered"
            args="load depth_image_proc/point_cloud_xyzrgb $(arg manager) $(arg bond)" respawn="$(arg respawn)">
        <remap from="rgb/image_rect_color"        to="$(arg rgb)/image_rect_color" />
        <remap from="rgb/camera_info"             to="$(arg rgb)/camera_info" />
        <remap from="depth_registered/image_rect" to="$(arg depth_registered_filtered)/sw_registered/image_rect_raw" />
        <remap from="depth_registered/points"     to="$(arg depth_registered)/points" />
      </node>
    </group>

    <group if="$(eval depth_registered_processing and hw_registered_processing)">
      <!-- Publish registered XYZRGB point cloud with hardware registered input (ROS Realsense depth alignment) -->
      <node pkg="nodelet" type="nodelet" name="points_xyzrgb_hw_registered"
            args="load depth_image_proc/point_cloud_xyzrgb $(arg manager) $(arg bond)" respawn="$(arg respawn)">
        <remap from="rgb/image_rect_color"        to="$(arg rgb)/image_rect_color" />
        <remap from="rgb/camera_info"             to="$(arg rgb)/camera_info" />
        <remap from="depth_registered/image_rect" to="$(arg depth_registered)/image_raw" />
        <remap from="depth_registered/points"     to="$(arg depth_registered_pub)/points" />
      </node>
    </group>
  </group>

  <group ns="$(arg camera3)">
    <!-- Launch the camera device nodelet-->
    <include file="$(find realsense2_camera)/launch/includes/nodelet.launch.xml">
      <arg name="external_manager"         value="$(arg external_manager)"/>
      <arg name="manager"                  value="$(arg manager)"/>
      <arg name="tf_prefix"                value="$(arg tf_prefix_camera3)"/>
      <arg name="serial_no"                value="$(arg serial_no_camera3)"/>
      <arg name="usb_port_id"              value="$(arg usb_port_id)"/>
      <arg name="device_type"              value="$(arg device_type)"/>
      <arg name="json_file_path"           value="$(arg json_file_path)"/>

      <arg name="enable_pointcloud"        value="$(arg enable_pointcloud)"/>
      <arg name="enable_sync"              value="$(arg enable_sync)"/>
      <arg name="align_depth"              value="$(arg align_depth)"/>

      <arg name="depth_width"              value="$(arg depth_width)"/>
      <arg name="depth_height"             value="$(arg depth_height)"/>
      <arg name="enable_depth"             value="$(arg enable_depth)"/>

      <arg name="infra_width"              value="$(arg infra_width)"/>
      <arg name="infra_height"             value="$(arg infra_height)"/>
      <arg name="enable_infra"            value="$(arg enable_infra)"/>

      <arg name="color_width"              value="$(arg color_width)"/>
      <arg name="color_height"             value="$(arg color_height)"/>
      <arg name="enable_color"             value="$(arg enable_color)"/>

      <arg name="depth_fps"                value="$(arg depth_fps)"/>
      <arg name="infra_fps"                value="$(arg infra_fps)"/>
      <arg name="color_fps"                value="$(arg color_fps)"/>
      <arg name="gyro_fps"                 value="$(arg gyro_fps)"/>
      <arg name="accel_fps"                value="$(arg accel_fps)"/>
      <arg name="enable_gyro"              value="$(arg enable_gyro)"/>
      <arg name="enable_accel"             value="$(arg enable_accel)"/>
      <arg name="filters"                  value="$(arg filters)"/>
      <arg name="unite_imu_method"         value="$(arg unite_imu_method)"/>
    </include>

    <!-- RGB processing -->
    <include if="$(arg rgb_processing)"
              file="$(find rgbd_launch)/launch/includes/rgb.launch.xml">
      <arg name="manager"                       value="$(arg manager)" />
      <arg name="respawn"                       value="$(arg respawn)" />
      <arg name="rgb"                           value="$(arg rgb)" />
      <arg name="debayer_processing"            value="$(arg debayer_processing)" />
    </include>

    <group if="$(eval depth_registered_processing and sw_registered_processing)">
      <node pkg="nodelet" type="nodelet" name="register_depth"
            args="load depth_image_proc/register $(arg manager) $(arg bond)" respawn="$(arg respawn)">
        <remap from="rgb/camera_info"             to="$(arg rgb)/camera_info" />
        <remap from="depth/camera_info"           to="$(arg depth)/camera_info" />
        <remap from="depth/image_rect"            to="$(arg depth)/image_rect_raw" />
        <remap from="depth_registered/image_rect" to="$(arg depth_registered)/sw_registered/image_rect_raw" />
      </node>

      <!-- Publish registered XYZRGB point cloud with software registered input -->
      <node pkg="nodelet" type="nodelet" name="points_xyzrgb_sw_registered"
            args="load depth_image_proc/point_cloud_xyzrgb $(arg manager) $(arg bond)" respawn="$(arg respawn)">
        <remap from="rgb/image_rect_color"        to="$(arg rgb)/image_rect_color" />
        <remap from="rgb/camera_info"             to="$(arg rgb)/camera_info" />
        <remap from="depth_registered/image_rect" to="$(arg depth_registered_filtered)/sw_registered/image_rect_raw" />
        <remap from="depth_registered/points"     to="$(arg depth_registered)/points" />
      </node>
    </group>

    <group if="$(eval depth_registered_processing and hw_registered_processing)">
      <!-- Publish registered XYZRGB point cloud with hardware registered input (ROS Realsense depth alignment) -->
      <node pkg="nodelet" type="nodelet" name="points_xyzrgb_hw_registered"
            args="load depth_image_proc/point_cloud_xyzrgb $(arg manager) $(arg bond)" respawn="$(arg respawn)">
        <remap from="rgb/image_rect_color"        to="$(arg rgb)/image_rect_color" />
        <remap from="rgb/camera_info"             to="$(arg rgb)/camera_info" />
        <remap from="depth_registered/image_rect" to="$(arg depth_registered)/image_raw" />
        <remap from="depth_registered/points"     to="$(arg depth_registered_pub)/points" />
      </node>
    </group>
  </group>
</launch>
@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jul 18, 2023 •
edited
Loading

Hi @taeyeonggim As you are using the align_depth parameter and not align_depth.enable, it looks as though you are using the ROS1 wrapper (Kinetic, Melodic, Noetic). Can you confirm if you are using ROS1 please?

If you are using ROS1, which version of the RealSense SDK (librealsense) and RealSense ROS wrapper are you using? Because development of the ROS1 wrapper has ceased, the most recent librealsense / wrapper combination that should be used with it is librealsense 2.51.1 and wrapper 2.3.2 as the ROS1 wrapper has not been updated for newer librealsense versions such as 2.53.1 and 2.54.1.

Can your three cameras be used with the ROS1 wrapper if they are launched from three separate ROS terminals instead of with a single launch, as described at the ROS wrapper documentation section linked to below.

https://github.com/IntelRealSense/realsense-ros/tree/ros1-legacy#work-with-multiple-cameras

For example:

roslaunch realsense2_camera rs_camera.launch camera:=cam_1 serial_no:=<serial number of the first camera>
roslaunch realsense2_camera rs_camera.launch camera:=cam_2 serial_no:=<serial number of the second camera>
roslaunch realsense2_camera rs_camera.launch camera:=cam_3 serial_no:=<serial number of the third camera>

@ghost
Copy link
Author

ghost commented Jul 19, 2023 •
edited by ghost
Loading

@MartyG-RealSense, Thank you for your quick reply. 👍
I'm currently using ROS1 melodic with align depth enable. Also librealsense, wrapper are using the most recent versions such as 2.51.1 and 2.3.2.

I am having issues with running 3 at once on ROS1, but not when running them one after the other.

Additionally, if we move to ROS2, will there be any issues with turning on multiple units at once?

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jul 19, 2023 •
edited
Loading

Have you tried the ROS1 launch file rs_multiple_devices.launch which supports three cameras, please?

https://github.com/IntelRealSense/realsense-ros/blob/ros1-legacy/realsense2_camera/launch/rs_multiple_devices.launch

roslaunch realsense2_camera rs_multiple_devices.launch serial_no_camera1:=<serial number of the first camera> serial_no_camera2:=<serial number of the second camera> serial_no_camera3:=<serial number of the third camera>

The ROS2 wrapper has its own multiple camera launch file called rs_multi_camera_launch.py, though this one is designed for only two cameras.

https://github.com/IntelRealSense/realsense-ros/blob/ros2-development/realsense2_camera/launch/rs_multi_camera_launch.py

@ghost
Copy link
Author

ghost commented Jul 19, 2023

I've already tried running rs_multiple_devices.launch, and it didn't fix the topic not coming up.
ROS2 also doesn't officially provide launch files for more than 3 cameras. 😢.

so I guess the only way to do that is to run the cameras sequentially, right?

Is librealsense the reason this is happening?

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jul 19, 2023 •
edited
Loading

The librealsense SDK is not a limitation on the number of cameras, as it is possible to list as many as 20 cameras in the RealSense Viewer (though not have 20 cameras enabled simultaneously due to the limitations of a computer's resources). It can certainly have four cameras active simultaneously with a powerful computer specification such as an i7 processor. Your i9 computer is easily a strong enough specification to process three cameras.

Problems with multiple camera setups do seem to occur more often in the ROS wrapper than they do in librealsense, such as the case at #1515 (comment) where six D435i cameras were being used and at least one would not publish topics.

If the ROS1 wrapper is being used and it was installed from packages with the wrapper's Method 1 instructions then there is the potential for multicam problems as the ROS1 packages are based on the RSUSB backend, which is not ideal for use with multiple cameras.

Building librealsense and the ROS wrapper separately from source code with the 'native backend' method (applying a patch script to the kernel and then building librealsense from source, followed by building the wrapper) would work better with multicam.

@MartyG-RealSense
Copy link
Collaborator

Hi @taeyeonggim Do you require further assistance with this case, please? Thanks!

@MartyG-RealSense
Copy link
Collaborator

Case closed due to no further comments received.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
D400 l500
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@MartyG-RealSense