Odroid XU4 + D455: No Device detected

[lprobsth]

Required Info
Camera Model	D455
Firmware Version	not sure / currently can't check
Operating System & Version	Ubuntu 16 / Ubuntu 18
Kernel Version (Linux Only)	4.14.200-? (U16 - not sure) / 4.14.180-178 (U18)
Platform	Odroid XU4
SDK Version	2.41.0
Language	-
Segment	Robot

Issue Description

I downloaded the GitHub repo of librealsense and followed the tutorial for manual installation on linux.

I did this one time on Ubuntu 16 with "patch-realsense-ubuntu-odroid.sh" and one time on Ubuntu 18 with "patch-realsense-ubuntu-odroid-xu4-4.14.sh".
Some patches failed in "patch-realsense-ubuntu-odroid.sh". I replaced the patch file "realsense-camera-formats-xenial-hwe.patch.txt" with the one from #7863 (comment) and then the patch worked.

In both versions oft Ubuntu and with no errors while patching all tools "rs-*" output "there are no connected devices".
When I open "realsense-viewer" it outputs in the console:

libGL error: failed to create dri screen
libGL error: failed to load driver: exynos
 20/01 10:18:14,241 WARNING [3070009360] (ds5-factory.cpp:1123) DS5 group_devices is empty.

I think the libGL error can be ignored...

Output of dmesg:

[144539.519037] usb 1-1: new high-speed USB device number 3 using exynos-ehci
[144539.676222] usb 1-1: New USB device found, idVendor=8086, idProduct=0b5c
[144539.676230] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[144539.676236] usb 1-1: Product: Intel(R) RealSense(TM) Depth Camera 455 
[144539.676241] usb 1-1: Manufacturer: Intel(R) RealSense(TM) Depth Camera 455 
[144539.677340] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 455  (8086:0b5c)
[144539.678711] uvcvideo: Unable to create debugfs 1-3 directory.
[144539.678872] uvcvideo 1-1:1.0: Entity type for entity Intel(R) RealSense(TM) Depth Ca was not initialized!
[144539.678887] uvcvideo 1-1:1.0: Entity type for entity Processing 2 was not initialized!
[144539.678899] uvcvideo 1-1:1.0: Entity type for entity Camera 1 was not initialized!
[144539.679342] input: Intel(R) RealSense(TM) Depth Ca as /devices/platform/soc/12110000.usb/usb1/1-1/1-1:1.0/input/input5
[144539.680864] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) Depth Camera 455  (8086:0b5c)
[144539.681460] uvcvideo: Unable to create debugfs 1-3 directory.
[144539.681669] uvcvideo 1-1:1.3: Entity type for entity Processing 7 was not initialized!
[144539.681679] uvcvideo 1-1:1.3: Entity type for entity Extension 8 was not initialized!
[144539.681686] uvcvideo 1-1:1.3: Entity type for entity Camera 6 was not initialized!

I think that the patching of the kernel for ubcvideo (and / or the uvcvideo module) is not working.

I've read about another method in the manual for NVidia Jetson Devices:

In order to build the SDK using the RSUSB method and avoid the kernel patching procedure, please refer to libuvc_installation.sh script for details.

So I installed with "libuvc_installation.sh" in Ubuntu 16 and the device gets recognized. Realsense-viewer crashed and I need some ROS packages that are only compatible with Ubuntu 18 so reinstalled the OS. Since then I didn't have time to recompile librealsense via this method (compiling takes ~ 2 h). I think it would be the best if the normal installation method would be working...
Since there is not much information about librealsense + Odroid XU4 here, I'm going to update this issue when I get the RSUSB method running on Odroid.

[MartyG-RealSense]

Hi @chaotic-bruno As a starting point in investigating your case, could you please try reloading the udev rules and then rebooting your computer, as described in the advice to an Odroid XU4 user in the link below:

#3244 (comment)

[lprobsth]

Hey @MartyG-RealSense,
I recompiled librealsense on Ubuntu 18 (kernel 4.14.180-178) with prior patching of the kernel.
I also checked the existence of the "99-realsense-libusb.rules" udev file and did sudo udevadm control --reload-rules && udevadm trigger && sudo reboot.

Nothing changed so far.

Here is the output of "realsense-viewer":

 20/01 18:54:17,796 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:2/device/../../../../../../../../../../
 20/01 18:54:17,796 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:0/device/../../../../../../../../../../
 20/01 18:54:17,796 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:6/device/../../../../../../../../../../
 20/01 18:54:17,798 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:4/device/../../../../../../../../../../
 20/01 18:54:17,798 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:3/device/../../../../../../../../../../
 20/01 18:54:17,799 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:1/device/../../../../../../../../../../
 20/01 18:54:17,799 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:7/device/../../../../../../../../../../
 20/01 18:54:17,800 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:5/device/../../../../../../../../../../
 20/01 18:54:17,817 INFO [3069165584] (backend-hid.cpp:1173) Failed to read busnum/devnum. Device Path: /sys/bus/iio/devices/iio:device0
 20/01 18:54:17,817 WARNING [3069165584] (ds5-factory.cpp:1123) DS5 group_devices is empty.
 20/01 18:54:17,818 INFO [3069165584] (context.cpp:381) Found 0 RealSense devices (mask 254)
 20/01 18:54:18,053 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:2/device/../../../../../../../../../../
 20/01 18:54:18,053 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:0/device/../../../../../../../../../../
 20/01 18:54:18,054 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:6/device/../../../../../../../../../../
 20/01 18:54:18,055 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:4/device/../../../../../../../../../../
 20/01 18:54:18,055 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:3/device/../../../../../../../../../../
 20/01 18:54:18,056 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:1/device/../../../../../../../../../../
 20/01 18:54:18,056 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:7/device/../../../../../../../../../../
 20/01 18:54:18,057 INFO [3069165584] (backend-v4l2.cpp:565) Failed to read busnum/devnum. Device Path: /sys/dev/char/81:5/device/../../../../../../../../../../
 20/01 18:54:18,074 INFO [3069165584] (backend-hid.cpp:1173) Failed to read busnum/devnum. Device Path: /sys/bus/iio/devices/iio:device0
 20/01 18:54:18,075 WARNING [3069165584] (ds5-factory.cpp:1123) DS5 group_devices is empty.
 20/01 18:54:18,075 INFO [3069165584] (context.cpp:381) Found 0 RealSense devices (mask 254)

Regarding the RSUSB method:
This works in Ubuntu 18 (device gets recognized + realsense-viewer starts) - but the connection without patched kernel is horribly slow (0.5 - 1 Hz). I will investigate this further - maybe I can tune this till I reach ~ 5 Hz in ROS.

I hope the method with the patched kernel will be faster...

[MartyG-RealSense]

The busnum/devnum error is a very rare one. In the past it has typically been related to use of a camera on a micro-size USB 3 OTG port on small developer boards and computers on a stick. The XU4 specification does not list an OTG port. However, on the affected devices the root cause was apparently insufficient power being provided to the camera by the USB port.

There was also a busnum/devnum case from December 2018 where the camera would work fine on SDK example programs but would disconnect when ROS was launched. One RealSense user found that if they encountered this problem with ROS then they could clear it by unplugging and re-inserting the camera a few times (though an initial_reset should have a similar effect to resetting the camera with an unplug-replug).

Are you usng the original official 1 meter USB cable that was supplied with the camera, please?

[lprobsth]

Thank you for your answer @MartyG-RealSense,

yes it is the original USB cable.

Yesterday I compiled librealsense with -DFORCE_LIBUVC=true (the build option from libuvc_installation.sh) and this does indeed work:

Regarding the RSUSB method:
This works in Ubuntu 18 (device gets recognized + realsense-viewer starts) - but the connection without patched kernel is horribly slow (0.5 - 1 Hz). I will investigate this further - maybe I can tune this till I reach ~ 5 Hz in ROS.

I hope the method with the patched kernel will be faster...

So i don't think that this is power related since I toggled the option in realsense-viewer for active IR and this should max out the power usage of the camera (see image below).

I will try now to build the librealsense2 ROS package with libuvc and see if the frame rate is sufficient for my use case.

[MartyG-RealSense]

Great to hear that you made progress @chaotic-bruno - please let me know how your next test with libuvc goes. Good luck!

[lprobsth]

Hey @MartyG-RealSense,

after some tuning I got it running in ROS melodic and the connection is fast and stable.
In the launch files of the realsense ROS package I had to enable the option <arg name="enable_sync" default="true"> or else I get this error:

 21/01 18:15:30,739 WARNING [2757739536] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: No data available, number: 61

as seen in: IntelRealSense/realsense-ros#1284 (comment)

We can close this issue if you don't need further information about the situation with the patched kernel ...

[MartyG-RealSense]

Great news that you succeeded @chaotic-bruno - thanks very much for sharing the method that worked for you! I do not require further information, so feel free to close this case if you are satisifed with the outcome.

[lprobsth]

Even though this is closed:

I tried mapping with rtabmap in ROS but the result was not good since the camera dropped frames for some seconds after 20-60 seconds of usage and even sometimes completely crashed. So I started searching again and found this blog https://jack-kawell.com/2019/09/30/installing-realsense-sdk-on-an-odroid-xu4/.

There it says before the compilation of librealsense:

Now before we build the package, we must first make a slight change to the source code to make it compatible with the d435i. This is due to the fact that the d435i has an integrated IMU which doesn’t work with the ODROID and causes the device to fail on connection. To make this change we’ll run the following command to edit the problem file:

gedit src/ds5/ds5-factory.cpp

Now in the gedit window scroll way down to the line that says all_sensors_present &= (hids.capacity() >= 2); and change the 2 to 1. And that’s it! Save the file and close the gedit window to return back to your terminal window.

I can confirm that this method works.

Comparison of the performance:

Compile Flag	Video Setting	rostopic hz /camer/aligned.../image_raw
-DFORCE_LIBUVC=true -DCMAKE_BUILD_TYPE=Release	424x240@60 FPS	2-4 Hz with massive dropped frames
-DCMAKE_BUILD_TYPE=Release	424x240@60 FPS	26-28 Hz

So the version working with patched kernel modules is indeed a lot faster. The downside is that the IMU is not usable.
As I want to use the camera on a humanoid robot while walking, the camera is shaked a lot during mapping - the knowledge of correct orientation of the camera is crucial. So I'm going to test the visual odometry with this performance of the depth camera. Maybe it's fast enough so rtabmap does its job.

[MartyG-RealSense]

It is excellent to hear that you have achieved improved results!

In regard to shaking during motion, you could perhaps add a vibration damping pad to the mounting point of the camera on the robot to reduce the impact of the motion.

#4636 (comment)

Using relocalization with RTABMAP to correct the IMU when tracking drifts or is lost has also been discussed. The link below has a tutorial that expands upon Intel's own D435i SLAM tutorial for ROS with the addition of how to use the localization mode of RTABMAP to obtain the relative position of the camera on a recorded map - see the tutorial section headed Localization with rtabmap.

https://shinkansan.github.io/2019-UGRP-DPoom/SLAM

[MartyG-RealSense]

Hi @chaotic-bruno You sent a note about whether you could re-open the case. If you wish to continue this case, you can re-open it with the Reopen Issue button underneath the comment box.

[lprobsth]

Hey @MartyG-RealSense,

1. I realized that in patch-realsense-ubuntu-odroid-xu4-4.14.sh the kernel modules for gyro and accel don't compile
2. Also in patch-realsense-ubuntu-odroid-xu4-4.14.sh the command for inserting the modules is commented out

First thing what I did was uncommenting the lines in patch-realsense-ubuntu-odroid-xu4-4.14.sh - then I tried to run patch-realsense-ubuntu-odroid-xu4-4.14.sh again. But the necessary modules didn't compile. So I ran the make commands by manually:

cd librealsense/odroid_bionic
KBASE=`pwd`
sudo make -j -C $KBASE M=$KBASE/drivers/iio/accel modules
sudo make -j -C $KBASE M=$KBASE/drivers/iio/gyro modules

But then again the modules did not recompile. This is due to the default config for the XU4 which was created by the command sudo make odroidxu4_defconfig. The necessary flags in the kernel config are "CONFIG_HID_SENSOR_ACCEL_3D" and "CONFIG_HID_SENSOR_GYRO_3D". Both were not set.
After setting these, one has to enter sudo make modules_prepare to update the config for dependencies (as far as I know).
Running make again on these modules resulted in a warning that some variables in the modules were not set.

I think this is because I was compiling these modules alone and not with their dependencies. Because I've run patch-realsense-ubuntu-odroid-xu4-4.14.sh before, all patches were already applied to the kernel source (patch-realsense-ubuntu-odroid-xu4-4.14.sh downloads the kernel from hardkernel/odroid and just compiles needed modules). So I just compiled the whole kernel - and this works!

Commands for compiling the modified kernel and activating it (compare to https://wiki.odroid.com/odroid-xu4/os_images/linux/ubuntu_4.14/20190929#linux_kernel_build):

make -j8
sudo make modules_install
sudo cp -f arch/arm/boot/zImage /media/boot
sudo cp -f arch/arm/boot/dts/exynos5422-odroid*dtb /media/boot
sync

Here is my version of patch-realsense-ubuntu-odroid-xu4-4.14.sh: https://pastebin.com/JgAq9DiS
Modifications:

• download specific version of the kernel from hardkernel (via uname -r)
• setting flags in kernel-".config" for the modules normally not included in the odroid kernel
• including insertion of the modified kernel modules

If one would kompile the whole kernel the dependencies of the accel and gyro kernel modules should be presend in the kernel. Then the patch-realsense-ubuntu-odroid-xu4-4.14.sh script should work - but I don't have time right now for testing this because I'm working on my thesis.

I hope this helps others when working with odroid devices...

[MartyG-RealSense]

Thanks so much @chaotic-bruno - that is an excellently detailed guide for Odroid XU4 developers! :)

[lprobsth]

Some further update:

The USB3.2 ports of the Odroid XU4 really have a power problem. The Camera stops working on them during heavy load.
Using the camera on the USB2 port works indefinitely.

In this post in the odroid forum there is explained how to clean the USB3 ports. But this didn't resolved my issue. I also tested several power supplies...

Other places where this issue was mentioned:
#3374 (comment)
https://www.reddit.com/r/ODroid/comments/cjc4g3/odroid_xu4_usb_3_ports_are_kicking_the_bucket/

[MartyG-RealSense]

Thanks again @chaotic-bruno for sharing your valuable XU4 test information!