No GPU usage on jetson xavier NX

[Camilochiang]

Hei. I recently move to a jetson xavier NX and tried to use GPU to get better results of alignment, but after checking with jtop there is 0 GPU usage and the aligned topic only give 1.5 FPS, meanwhile color and depth give the right values (in this case 15 FPS)

Here my steps:

#FIRMWARE
rs-fw-update -f Signed_Image_UVC_5_13_0_50.bin
# success

#LibRealSense
wget https://github.com/IntelRealSense/librealsense/archive/refs/tags/v2.50.0.zip
unzip v2.50.0.zip
rm v2.50.0.zip
cd librealsense-2.50.0
mkdir build
cd build
cmake -DFORCE_RSUSB_BACKEND=ON \
-DBUILD_PYTHON_BINDINGS=true \
-DPYTHON_EXECUTABLE=/usr/bin/python3.6 \
-DCMAKE_BUILD_TYPE=release \
-DBUILD_EXAMPLES=true \
-DBUILD_GRAPHICAL_EXAMPLES=true \
-DBUILD_WITH_CUDA=true ..
make -j4
sudo make install
# This in theory should use RSUSB backend
# We can confirm with any of the rs-functions
# rs-hello-realsense is working!

#ROS wrapper
cd /ROS/src
wget https://github.com/IntelRealSense/realsense-ros/archive/refs/tags/2.3.2.zip
unzip 2.3.2.zip
rm 2.3.2.zip
cd ..
catkin_init_workspace
cd ..
catkin_make clean
catkin_make -DCATKIN_ENABLE_TESTING=False -DCMAKE_BUILD_TYPE=Release
catkin_make install

source /ROS/devel/setup.bash
roslaunch realsense2_camera rs_camera.launch

Specs:

   * Jetpack 4.6 [L4T 32.6.1]
   * jetson_stats.service: active
   * RS firmware : 13.0.5
   * librealsense : 2.50.0
   * roswrapper : 2.3.2

I check This question but not sure why it may be failing. Any suggestions people? @MartyG-RealSense ?

[Camilochiang]

So after removing a previous installation (sudo apt remove ros-melodic-realsense2-camera), I'm getting the same error that has been previously reported by in the above mentioned question

//librealsense2_camera.so: undefined symbol: _ZN2cv3MatC1Ev

[MartyG-RealSense]

Hi @Camilochiang It is a known issue that align and pointcloud can have problems such as heavy FPS lag or missing color on Jetson boards specifically. The problem does not usually occur on non-Jetson computers such as desktop and laptop PCs. #1967 discusses the Jetson pointcloud problem, though the align problem began at around the same time.

A couple of RealSense ROS users in that case resolved the pointcloud issue using librealsense 2.43.0 and wrapper 2.2.23 - as described at #1967 (comment) - so it may be worth trying that configuration to see whether it resolves the align problem too, since it began at around the same time as the pointcloud problem.

Currently, using the older configuration described in #1967 (comment) is the best known solution.

In regard to the error undefined symbol: _ZN2cv3MatC1Ev, if you have built the ROS wrapper from source then can you make sure that you have ddynamic_reconfigure installed by using the command below, please:

sudo apt install ros-melodic-ddynamic-reconfigure

[Camilochiang]

Hei @MartyG-RealSense . Yes, I had the ros-melodic-ddynamic-reconfigure module installed.

I retry with the following configuration and no success. Error undefined symbol: _ZN2cv3MatC1Ev still present

cd
cd ASPEN/Software/transfer/
rs-fw-update -f Signed_Image_UVC_5_12_12_100.bin

#LibRealSense
cd /home/aspen/Others
sudo rm -r librealsense-2.50.0
#wget https://github.com/IntelRealSense/realsense-ros/archive/refs/tags/v2.43.0.zip
# had to do it manually
unzip librealsense-2.43.0.zip
rm librealsense-2.43.0.zip
cd librealsense-2.43.0
mkdir build && cd build
cmake -DFORCE_RSUSB_BACKEND=ON \
-DBUILD_PYTHON_BINDINGS=true \
-DPYTHON_EXECUTABLE=/usr/bin/python3.6 \
-DCMAKE_BUILD_TYPE=release \
-DBUILD_EXAMPLES=true \
-DBUILD_GRAPHICAL_EXAMPLES=true \
-DBUILD_WITH_CUDA=true ..
make -j4
sudo make install

#Checking dynamic presence
sudo apt install ros-melodic-ddynamic-reconfigure

#Wrapper
cd /home/aspen/ASPEN/Software/ROS/src
wget https://github.com/IntelRealSense/realsense-ros/archive/refs/tags/2.2.23.zip
unzip 2.2.23.zip
rm 2.2.23.zip
catkin_init_workspace
cd ..
catkin_make clean
catkin_make -DCATKIN_ENABLE_TESTING=False -DCMAKE_BUILD_TYPE=Release
catkin_make install

source /home/$USER/ASPEN/Software/ROS/devel/setup.bash
roslaunch realsense2_camera rs_camera.launch

Specs:

- Jetpack 4.6 [L4T 32.6.1]
- Opencv 4.5.3 
- firmware 5.12.12.100 
- lib 2.43.0 
- wrapper 2.2.23

[Camilochiang]

Something that call my attention is that meanwhile installing opencv from source all libraries were installed to /usr/local/lib/aarch64-linux-gnu, for example:

sudo make install
...
-- Set runtime path of "/usr/local/lib/aarch64-linux-gnu/libopencv_core.so.4.5.3" to "/usr/local/lib/aarch64-linux-gnu"
-- Set runtime path of "/usr/local/lib/aarch64-linux-gnu/libopencv_flann.so.4.5.3" to "/usr/local/lib/aarch64-linux-gnu"
-- Set runtime path of "/usr/local/lib/aarch64-linux-gnu/libopencv_imgproc.so.4.5.3" to "/usr/local/lib/aarch64-linux-gnu"
-- Set runtime path of "/usr/local/lib/aarch64-linux-gnu/libopencv_ml.so.4.5.3" to "/usr/local/lib/aarch64-linux-gnu"

Meanwhile when installing the ros-wrapper with catkin_make install, librealsense2_camera.so doesnt have a path...

-- Set runtime path of "/home/aspen/ASPEN/Software/ROS/install/lib/librealsense2_camera.so" to ""

I suspect that manually coping that file to /usr/local/lib/python3.6/dist-packages/cv2/python-3.6 may solve the problem. I will have a look

[MartyG-RealSense]

Okay, please do let me know how your manual file-copying goes.

[koralm]

Camilochiang Maybe I found a solution try adding/modifying to CMakeLists in realsense2_camera. Now im testing this solution but it looks promising. (even work with -DFORCE_RSUSB_BACKEND:=false):

1. find_package( OpenCV REQUIRED )
2. include_directories(
   include
   ${realsense2_INCLUDE_DIR}
   ${catkin_INCLUDE_DIRS}
   ${OpenCV_INCLUDE_DIRS}
   )
3. target_link_libraries(${PROJECT_NAME}
   ${realsense2_LIBRARY}
   ${catkin_LIBRARIES}
   ${CMAKE_THREAD_LIBS_INIT}
   ${OpenCV_LIBRARIES}
   )

[MartyG-RealSense]

Thanks so much @koralm for sharing your possible solution with @Camilochiang and the RealSense ROS community :)

[Camilochiang]

Hei @MartyG-RealSense . The suggested solution from @koralm did the trick. I guess that should also work with more recent versions but so far is working (more than 15 aligned FPS) in:

• jetson xavier NX: Jetpack 4.6 [L4T 32.6.1]
• Opencv 4.5.3
• Realsense firmware 5.12.12.100
• librealsense 2.43.0
• ros wrapper 2.2.23
  I will tested in the newer versions of librealsense and wraper later (also other version of openCv)

[MartyG-RealSense]

It's excellent to hear that the solution of @koralm worked for you - thanks very much for the update!

[edgarcamilocamacho]

Hello @Camilochiang and @MartyG-RealSense. I'm facing the same problem but using ROS Galactic, I can change the firmware and the librealsense versions you say, but what about the 2.2.23 wrapper version in ROS2?

[MartyG-RealSense]

Hi @edgarcamilocamacho The 2.2.23 wrapper is for ROS1. The equivalent ROS2 wrapper for matching to SDK 2.43.0 is 3.1.5 but ROS2 Galactic support was not introduced until wrapper 3.2.2 which is designed for matching with SDK 2.48.0.

[edgarcamilocamacho]

Thanks, @MartyG-RealSense. I am using that one right now, and modified the CMakeList to accept galactic. Just some problems with TF naming, but after manually setting them it is working. It can be a temporal solution while one the 3.X.X or 4.X.X version is fixed.

[MartyG-RealSense]

Thanks so much @edgarcamilocamacho for sharing your workaround solution for ROS2 with the RealSense ROS community!

[robotom]

Camilochiang Maybe I found a solution try adding/modifying to CMakeLists in realsense2_camera. Now im testing this solution but it looks promising. (even work with -DFORCE_RSUSB_BACKEND:=false):

1. find_package( OpenCV REQUIRED )

2. include_directories(
   include
   ${realsense2_INCLUDE_DIR}
   ${catkin_INCLUDE_DIRS}
   ${OpenCV_INCLUDE_DIRS}
   )

3. target_link_libraries(${PROJECT_NAME}
   ${realsense2_LIBRARY}
   ${catkin_LIBRARIES}
   ${CMAKE_THREAD_LIBS_INIT}
   ${OpenCV_LIBRARIES}
   )

I tried this and I'm getting the same symbol issue error. I'm using an AGX Orin. Does anyone have any ideas?

Here's my cmakelists:

Click to expand/collapse

cmake_minimum_required(VERSION 2.8.3)
project(realsense2_camera)
add_compile_options(-std=c++11)

find_package( OpenCV REQUIRED )

option(BUILD_WITH_OPENMP "Use OpenMP" OFF)
option(SET_USER_BREAK_AT_STARTUP "Set user wait point in startup (for debug)" OFF)

add_definitions(-D_CRT_SECURE_NO_WARNINGS)
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)

find_package(catkin REQUIRED COMPONENTS
message_generation
nav_msgs
roscpp
sensor_msgs
std_msgs
std_srvs
nodelet
cv_bridge
image_transport
tf
ddynamic_reconfigure
diagnostic_updater
)

if(BUILD_WITH_OPENMP)
find_package(OpenMP)
if(NOT OpenMP_FOUND)
message(FATAL_ERROR "\n\n OpenMP is missing!\n\n")
else()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS} -fopenmp")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${OpenMP_EXE_LINKER_FLAGS}")
endif()
endif()

if(SET_USER_BREAK_AT_STARTUP)
message("GOT FLAG IN CmakeLists.txt")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -DBPDEBUG")
endif()

if (WIN32)
find_package(realsense2 CONFIG REQUIRED)
else()
find_package(realsense2 2.50.0)
endif()

if(NOT realsense2_FOUND)
message(FATAL_ERROR "\n\n Intel RealSense SDK 2.0 is missing, please install it from https://github.com/IntelRealSense/librealsense/releases\n\n")
endif()

if(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE Release)
endif()

string(TOUPPER "${CMAKE_BUILD_TYPE}" uppercase_CMAKE_BUILD_TYPE)
if (${uppercase_CMAKE_BUILD_TYPE} STREQUAL "RELEASE")
message(STATUS "Create Release Build.")
set(CMAKE_CXX_FLAGS "-O2 ${CMAKE_CXX_FLAGS}")
else()
message(STATUS "Create Debug Build.")
endif()

if (WIN32)
else()
set(CMAKE_CXX_FLAGS "-fPIE -fPIC -std=c++11 -D_FORTIFY_SOURCE=2 -fstack-protector -Wformat -Wformat-security -Wall ${CMAKE_CXX_FLAGS}")
endif()

add_message_files(
FILES
IMUInfo.msg
Extrinsics.msg
Metadata.msg
)

add_service_files(
FILES
DeviceInfo.srv
)

generate_messages(
DEPENDENCIES
sensor_msgs
std_msgs
)

set(CMAKE_NO_SYSTEM_FROM_IMPORTED true)
include_directories(
include
${realsense2_INCLUDE_DIR}
${catkin_INCLUDE_DIRS}
${OpenCV_INCLUDE_DIRS}
)

RealSense ROS Node

catkin_package(
INCLUDE_DIRS include
LIBRARIES ${PROJECT_NAME}
CATKIN_DEPENDS message_runtime roscpp sensor_msgs std_msgs
nodelet
cv_bridge
image_transport
ddynamic_reconfigure
nav_msgs
)

add_library(${PROJECT_NAME}
include/realsense2_camera/constants.h
include/realsense2_camera/realsense_node_factory.h
include/realsense2_camera/base_realsense_node.h
include/realsense2_camera/t265_realsense_node.h
src/realsense_node_factory.cpp
src/base_realsense_node.cpp
src/t265_realsense_node.cpp
)

add_dependencies(${PROJECT_NAME} ${PROJECT_NAME}_generate_messages_cpp)
add_dependencies(${PROJECT_NAME} ${catkin_EXPORTED_TARGETS})

target_include_directories(${PROJECT_NAME}
PRIVATE ${realsense2_INCLUDE_DIR})

target_link_libraries(${PROJECT_NAME}
${realsense2_LIBRARY}
${catkin_LIBRARIES}
${CMAKE_THREAD_LIBS_INIT}
${OpenCV_LIBRARIES}
)

if(WIN32)
set_target_properties(${realsense2_LIBRARY} PROPERTIES MAP_IMPORTED_CONFIG_RELWITHDEBINFO RELEASE)
target_link_libraries(${PROJECT_NAME}
realsense2::realsense2
realsense2::realsense-file
)
endif()

Install nodelet library

install(TARGETS ${PROJECT_NAME}
ARCHIVE DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
LIBRARY DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
RUNTIME DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
)

Install header files

install(DIRECTORY include/realsense2_camera
DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION}
)

Install launch files

install(DIRECTORY launch/
DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}/launch
)

Install rviz files

install(DIRECTORY rviz/
DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}/rviz
)

Install xml files

install(FILES nodelet_plugins.xml
DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}
)

EDIT: Perhaps I was a bit hasty to complain. However, this problem comes and goes for whatever reason. Sometimes it works, sometimes it doesn't. I have no idea why. Though, it only started to kind of work after I made this OpenCV modification. Any ideas?

[FPSychotic]

why this not included by default?