Stereo camera failure on D457 #13630
Comments
|
Hi @Tsubasa-Swarm If you are using the D457 camera with a USB cable, have you set the physical mode switch inside a hidden recess in the the camera casing for use with USB please? A D457 cannot be detected on USB unless this is done as it is configured for use with a different cable type called GMSL by default. Instructions for setting the mode switch can be found at the link below. https://support.intelrealsense.com/hc/en-us/community/posts/14840675121043/comments/14843232242835 |
|
Hello @MartyG-RealSense, I'm with the same team as @Tsubasa-Swarm, we have set the switch to USB mode and the D457 can be read through USB in Intel RealSense Viewer. Both the RGB Camera and Motion Module from the left pane can work. However, when we try to use the Stereo Module, some errors show up and the D457 gets disconnected from PC then reconnected (audible windows USB disconnect and reconnect sounds [but it's not disconnected physically]), but Stereo Module doesn't turn on and doesn't work. It's noteworthy to mention that the D457's Stereo Module was working before, but then it stopped working and it's also noteworthy to mention that we did a factory reset using the Intel RealSense Dynamic Calibration Tool and the "Intel.Realsense.CustomRW -g" command. we are trying to use Intel Real Sense Viewer to understand what the error is, debug it, and hopefully get the D457 working again with no errors. I attached some screenshot and a video that I hope will be of help. 20241227_014859_1.mp4 |
|
Have you tried unplugging the micro-sized end of the USB cable from the base of the camera, turning the connector around the other way and plugging it back into the camera? On the D457 camera model, the micro-sized end of the cable needs to be plugged into the camera in one particular direction of the two insertion directions available (the micro-sized end has two-way insertion), otherwise the camera operates in USB 2.1 mode like in the images above. |
|
I was previously using a USB2.0 port and USB-A to USB-C cable to connect the D457 to the PC, I reversed the USB-C connector from the D457's end as you said, but I still got the same behavior (in USB2.1 mode) with both connector orientations. Even though the D457 was able to work previously in this mode, I have used another USB-C to USB-C (for USB3.0) cable and now it's running in USB 3.2 mode, and as shown in the following videos (video size too big to be directly embedded here), a new behavior can be observed. Even though the depth module still can not work, when I turn it on now the camera doesn't get disconnected but I still get errors. Additionally, I wanted to note that I'm also using a Jetson Orin Nano (Jetpack 6.0) with Ubuntu 22.04.5 LTS to debug the D457. I have built the librealsense from source and installed it, I have also installed the RealSense utilities package (all of this has been done previously before the camera's depth module ceased working, the camera worked before with the same Jetson Orin Nano Environment).
I tried many things such as rebuilding Librealsense with RSUSB Backend (since I read that this should bypass V4L2 issues [if they exist]), loaded previous D457 firmware, .... However, I noticed something, when I run rs-enumerate-devices, I get errors and the camera disconnects, here are the details:
swarm@swarm-desktop:~/librealsense$ sudo dmesg | tail -n 20 swarm@swarm-desktop:~/librealsense$ lsusb -t
swarm@swarm-desktop:~/librealsense$ rs-enumerate-devices Finally, I wanted to also note that the camera's depth module (RGB and Motion modules are OK) stopped working in RealSense Viewer (windows) [and the camera with all its modules stopped being displayed in RealSense Viewer's source pane (Orin Nano's Ubuntu)] after I removed its casing (It's reattached now). I'm not sure if it stopped working directly after I removed the casing so I can't say removing the casing caused all the issues. Moreover, is it even possible that removing the casing will cause a hardware issue that impedes the depth module from working without affecting the RGB and Motion Modules. And if the issue is a hardware issue pertaining to USB, will using GMSL with FAKRA cable solve all connection issues or is the D457's motion module bricked because I opened the casing and someone from the team might have mistakenly caused a short (or other issue) while manipulating the Vision Processor D4 Board? I hope that this is not the case :-) |
|
Thanks very much for the very detailed information! Both video and screenshot are fine. I would not recommend using USB C to C cables if possible as they typically have more problems when used with RealSense cameras compared to A to C (also called USB Type C), which is the cable type supplied with RealSense camera models other than D457. Whilst D457 cameras can be used with GMSL / Fakra cables on Jetson boards instead of USB, it requires the purchase of a separate hardware module called a deserializer in order to use the D457 with that cable type. Simply removing the D457's casing may not cause a problem, though if the short 'interposer' cable connecting the two circuit boards together (the D450 depth board and Vision Processor D4 V5 board) became partially dislodged from one of its connection ports then that could cause depth to malfunction whilst still enabling the camera to be detected as D455. Opening the camera casing up will likely have invalidated the sales warranty and prevent the camera from being returned, so there is no harm now in opening the casing again and checking that both ends of the interposer cable are fully seated in their connection ports. However, I would strongly recommend testing on Windows with a USB 3 A to C cable first as such a cable will be inexpensive (or free if you already have one somewhere). Then if that still does not work, you could consider replacing the D4 V5 board, - part number 82635DSD457ASIC- which costs around 80 euros from an Intel-approved supplier such as Rutronik24. The D4 V5 board is the board that the USB cable connects onto. https://www.rutronik24.co.uk/product/intel/99c2m4/20357915.html |
|
Thank you very much for the recommendations. I bought a new USB A to C 10Gbps Cable for USB 3.2 mode.
However, unfortunately, even though the D457 is being read in USB 3.2 mode in RealSense Viewer [Windows] (using the USB A to C cable). I still get the same behavior as with USB C to C 3.2 mode. https://drive.google.com/file/d/1Sc5jWX2msFypxKrIiLCoei5zS4BeJ1ey/view?usp=sharing This is the full log from RealSense Viewer [Windows]: I also get the same behavior as described in my previous comment with the Jetson Orin Nano (Jetpack 6.0) [Ubuntu 22.04.5 LTS].
if the interposer cable becomes partially dislodged wouldn't that cause the RGB module to malfunction as well?
I can confirm that the interposer cable is fully plugged into the depth module receptacle.
Is there anything else I can do to test or check what the exact problem is? are we sure that this malfunction has been isolated to a hardware issue and not a software one? If it's probably a hardware issue and there's no need for further checking, which course of action do you recommend? Action plan 1:
Action plan 2:
Action plan 3: something else. |
|
Thanks so much for the detailed feedback and images! If the interposer is dislodged from its connection port then it typically affects depth rather than RGB. If the camera works correctly in the RealSense Viewer tool with an A to C cable but not a C to C, that suggests that the problem is with the C to C cable rather than the camera (as RealSense cameras often do not work well with C to C). The best place to ask about how to port the MIPI driver would be at the driver's own dedicated support forum at the link below, where Intel RealSense team members who are expert in this subject can provide advice. https://github.com/IntelRealSense/realsense_mipi_platform_driver/issues I would recommend investigating whether the C to C cable is the source of the problem. C to C cables can have quality issues, so obtaining a high quality USB 3.1 C to C cable could help to determine whether the current C to C cable you are using has a quality problem. |
I have so far used both high quality USB 3.2 C to C and USB 3.2 A to C cables and got the D457 to be read in USB 3.2 mode in RealSense Viewer [Windows]. However, I still get the same issue and the same behavior even with the USB 3.2 A to C cable as described in my previous comment. |
|
If you downgrade your camera firmware version from 5.16.0.1 to the previous version 5.15.1.0 then it could help to establish two things:
Instructions for performing the firmware change manually can be found at #12919 (comment) |
|
After physically disconnecting and reconnecting USB multiple times (as shown in the following log), I was able to downgrade through rs-fw-update tool from firmware version 5.16.0.1 to 5.15.1.0. rs-fw-update log: C:\Program Files (x86)\Intel RealSense SDK 2.0\tools>rs-fw-update -l Connected devices: C:\Program Files (x86)\Intel RealSense SDK 2.0\tools>rs-fw-update -l Connected devices:
C:\Program Files (x86)\Intel RealSense SDK 2.0\tools>rs-fw-update -l Connected devices: C:\Program Files (x86)\Intel RealSense SDK 2.0\tools> Connected devices:
C:\Program Files (x86)\Intel RealSense SDK 2.0\tools>rs-fw-update -f Signed_Image_UVC_5_15_1_0.bin C:\Program Files (x86)\Intel RealSense SDK 2.0\tools>rs-fw-update -f Signed_Image_UVC_5_15_1_0.bin Updating device: Firmware update started Firmware update progress: 100[%] Firmware update done Device 234222302258 successfully updated to FW: 5.15.1 C:\Program Files (x86)\Intel RealSense SDK 2.0\tools> The same malfunctioning behavior persists in RealSense Viewer afterwards. This is the full log: 22:22:13 [Info] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\context.cpp:336 - Found 1 RealSense devices (mask 0xff) 22:22:16 [Info] - Starting streaming of Depth... 22:22:23 [Error] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\global_timestamp_reader.cpp:239 - Error during time_diff_keeper polling: hr returned: HResult 0x80004002: "No such interface supported" 22:22:27 [Error] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\ds\d400\d400-thermal-monitor.cpp:94 - Error during thermal compensation handling: query is available during streaming only |
|
Has this D457 ever worked correctly since it was purchased, please? If it is very difficult for the computer to detect it, you could test it with a mains electricity powered USB 3.1 hub instead of plugging it directly into the computer if you have not attempted this already. It could eliminate the possibility that the computer's USB port is providing insufficient power for the camera to operate. A camera plugged into the computer's USB port will be drawing electricity from the computer, whilst a powered hub draws the electricity from the mains plug socket and so can be much more stable in its power delivery to the camera. |
Yes, I purchased it around March 2024 if I'm not mistaken, the issues started about 2 to 3 weeks ago.
The computer (and RealSense Viewer) can detect it and the RGB module and the motion module work correctly. It is only when I turn on the depth module that connection issues start. Should I still test with a USB hub just in case? (I'm also leaning towards this being a current draw issue although I'm not sure why it would start after using the D457 for so long). |
|
It would be worth testing with a powered hub if you have the option to do so in order to eliminate the possibility that the computer is providing insufficient power. Since the Vision Processor D4 board handles the power receival and distribution of the camera, there is also the possibility that the D4 V5 has been recently damaged in some way, such as an electrical power surge on the USB system. Excessive power could be caused by a glitch on the computer's USB port or internal damage in the USB cable (which could be caused by repeated cable bending). |
I will order powered USB Hub that supports 10 Gbps data transfer rate today and test.
Would a USB failure of this kind cause the depth module to malfunction without affecting the RGB and motion modules? |
|
If it was a case of insufficient power delivery, possibly, if the depth stream has higher power draw than RGB and Motion Module streams. A way to test it may be to set the Laser Power slider to zero in the Stereo Module section of the Viewer's options side-panel whilst the Stereo Module is disabled, and then enable it to start the depth stream. If depth starts and then you increase Laser Power and find that the camera disconnects, that would be strong evidence of a USB power related issue.. |
I tested with the powered USB Hub and still got the same error with the depth module. 2025-01-01.20-49-15.mp4
I tested this as well and the issue persists. 2025-01-01.20-51-32.mp4It's worth noting that sometimes when I set the Laser Power to 0 and start the depth stream the Projector Temperature field or the Asic Temperature field goes to a very high value as seen here: https://drive.google.com/file/d/15MeGjwlGnvZgHHhv814O_hQGZUz4Ht6q/view?usp=sharing It's also worth noting that even though I still get the same error with rs-enumerate-devices on the Jetson Orin Nano, when I tested rs-enumerate-devices on windows I get this: C:\Program Files (x86)\Intel RealSense SDK 2.0\tools>rs-enumerate-devices Stream Profiles supported by Stereo Module Stream Profiles supported by RGB Camera Stream Profiles supported by Motion Module C:\Program Files (x86)\Intel RealSense SDK 2.0\tools> |
|
The results from rs-enumerate-devices look normal. Have you tried any example programs other than realsense-viewer to cofirm whether the problem is with that program only? If you have installed the full SDK on Windows with the Intel.RealSense.SDK-WIN10 installer program then you can right-click on the RealSense Viewer launch shortcut icon on the desktop and select Open file location from the drop-down menu to go to the SDK's "Tools" folder, which contains executable versions of the RealSense example programs. In the Tools folder, if you double-leftclick on rs-capture to launch it then it should display the depth, RGB and IMU streams like in realsense-viewer. If you try this, is rs-capture able to display the depth stream? |
I have tried a simple python program on the Jetson Orin Nano (This program worked previously however after I started getting the depth module issue it stopped working too). here's the code and exec log: Code: import pyrealsense2 as rs config = rs.config() # カラーストリームを設定 # デプスストリームを設定 pipeline = rs.pipeline() pipeline.start(config) try: finally: Exec log: /bin/python /home/swarm/Python/realsense_study/distance_check.py
I have tested with rs-capture, rs-enumerate-devices, rs-hello-realsense, rs-measure, rs-motion and rs-pointcloud. Only rs-enumerate-devices and rs-motion worked as you can see here (I didn't physically disconnect D457 during this test): https://drive.google.com/file/d/1hb8D681bwh01JrphvEseNb5f_sZCSqml/view?usp=sharing |
|
In order for depth frames to be generated, infrared has to be working because a depth frame is produced from raw left and right infrared frames. If you deselect Depth in the Stereo Module section of the Viewer and select the Infrared stream, is it able to be displayed when the Stereo Module is enabled? |
I tried this and other configurations but still get the same error: https://drive.google.com/file/d/1lF8fBjxwEl8LpRncx44sEdy2nYMN3fD0/view?usp=sharing I have also tried turning off the laser emitter: 2025-01-02.22-18-56.mp4 |
|
All of the functions of the D4 V5 board seem to be working correctly, including the firmware driver and firmware update. The problems only seem to occur when trying to enable functions related to the left and right sensors on the D450 depth module board, which fail when attempts to enable them are made. The left and right sensors provide the depth and infrared images. Having tested the camera thoroughly on both Windows and Jetson computers, the evidence points towards a hardware failure on the D450 depth module. Given the cost of replacing a D450 and the issue of the current camera being non-returnable, it may be worth considering replacing it with a D456, which has the same capabilities and environmentally protected casing as D457 except for not having a GMSL connector. |
If there are no further tests needed and the issue has been isolated to a hardware failure on the D450 depth board. Then, considering that I have found a D450 board about half the price of a D435 and even lower compared to a D455, and considering that I want to fix the current D457 if possible, would you recommend replacing the current D450 board with the one available here: https://www.mouser.jp/ProductDetail/Intel/82635DSD450?qs=hd1VzrDQEGjg%252BAsSbWbygA%3D%3D Moreover, I have tried to disconnect the interposer from the D450's side in case the latter can be replaced with the board linked above but I wasn't successful in doing so. First, there was this thermal paste or some kind of paste or strong adhesive holding the D450 to the heat sink. I didn't want to physically force it too much so it doesn't break. you can see it here:
Second, I don't yet see a way of disconnecting the interposer from the D450 board without breaking the plastic part on top of it. Is there some way I can properly disconnect the interposer from the D450 board or is it not meant to be disconnected?
Finally, If the D450 can be properly disassembled, then is it possible to fix the board itself and perhaps replace certain faulty parts instead of replacing the whole board? |
|
Both of the boards in the camera (the D4 V5 and D450) can be swapped with a replacement. Mouser are an Intel approved supplier and are a highly reputable and reliable company to buy from. If there are customs fees for importing from another country then there may be additional costs on top of the purchase price. The boards are not intended to be repairable with replacement of individual components on them though. Yes, you should be able to connect and disconnect the interposer cable. If you cannot remove the cable from the D450, if you can disconnect it from the D4 V5's connector then there would be the option of purchasing another interposer cable from the official RealSense store and connecting it to the replacement D450. https://store.intelrealsense.com/buy-hsf50-flexi-interposer-cable.html I do not have personal hands-on experience of disassembly and reassembly of camera parts, so do not have advice to offer about the best way to do it. Thermal paste on electronics such as CPU chips can be dissolved using a special type of alcohol called isopropyl alcohol, also known as rubbing alcohol, and is safe to use on electronics. The paste is usually used for cleaning off paste from surfaces once a component has been removed though. A heatsink's bond should be breakable by making a gentle twisting motion a few degrees clockwise and anticlockwise. Overall, once all the possible little costs of repair (paste dissolver, replacement interposer cable, customs import fees, etc) add up then it may not be much cheaper than simply replacing the whole camera. |
I was able to fully disassemble the camera as you can here (I already had some isopropyl alcohol lying around).
I was also able to disconnect the interposer from the D450's side. It wasn't as clean as I wanted it to be but it worked nonetheless. The duty and customs fees are paid by mouser and the sales tax is about 3,200 JPY, so the D450 replacement board should total about 35,200 JPY as you can see here:
From a budget perspective, I think it's way cheaper than ordering, let's say a new D455, which costs about 73,500 JPY (before tax) as you can see here:
And it's still cheaper than ordering a new D435 which costs about 55,100 JPY (before tax) as you can see here:
Considering all of the above, and just to reconfirm, would you say that the D450 board available here (https://www.mouser.jp/ProductDetail/Intel/82635DSD450?qs=hd1VzrDQEGjg%252BAsSbWbygA%3D%3D) is fully compatible with the D4 V5 board I currently have and that you'd recommend that I go ahead with purchasing the D450 replacement from mouser instead of buying a new depth camera and fully discarding the D457? |
|
Yes, a D450 depth module board will be able to be matched with a D4 V5 board to produce a D457 camera configuration. The firmware driver is stored on the D4 V5 board, so once the two boards are joined then the firmware version should be 5.15.1.0, based on the most recent firmware update that you did. If you are located in Japan and are ordering from Mouser's Japanese website and are happy with the D450 board price and delivery fees then I see no reason why you cannot proceed with the purchase. |
|
Thank you very much for the confirmation. I'll update you once the D457 has been reassembled and tested with the new D450 replacement. Thanks again. |
When I connect the D457 to a Windows PC via USB 3.2 and test the video output with realsense-viewer, only the 2D RGB camera can be output, not the stereo camera. When the stereo camera is toggled on, the following error message appears repeatedly and the connection is broken. I would be grateful if you could think of a solution.
19:45:02 [Info] - No online SW / FW updates available
19:45:02 [Debug] - Current FW >= Bundled FW for: Intel RealSense D455 (S/N 234222302258)
Current Version: 5.16.0.1
Recommended Version: 5.16.0.1
19:45:04 [Info] - Starting streaming of Color...
19:45:04 [Info] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\proc\formats-converter.cpp:267 - Request: RGB8 Color,
Resolved to: YUYV Color,
19:45:10 [Info] - Starting streaming of Depth...
↓
19:45:14 [Info] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\uvc-sensor.cpp:163 - Frame counter reset
19:45:14 [Error] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\rs.cpp:237 - [rs2_allocate_synthetic_video_frame( source:00000207D2F10620, new_stream:00000207D2E9EF70, original:00000207D47CD098, new_bpp:3, new_width:1280, new_height:720, new_stride:3840, frame_type:Video Frame ) Wrong Api Call Sequence] Out of frame resources!
19:45:14 [Error] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\proc\synthetic-stream.cpp:58 - Exception was thrown during callback: Out of frame resources!
↑Repeatedly output this error
19:45:15 [Error] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\global_timestamp_reader.cpp:239 - Error during time_diff_keeper polling: hr returned: HResult 0x800703e3: "スレッドの終了またはアプリケーションの要求によって、I/O 処理は中止されました。"
19:45:17 [Error] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\error-handling.cpp:99 - Error during polling error handler: hr returned: HResult 0x800701b1: "存在しないデバイスを指定しました。"
19:46:01 [Error] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\error-handling.cpp:99 - Error during polling error handler: hr returned: HResult 0x800706bf: "リモート プロシージャ コールに失敗し、実行されませんでした。"
19:46:05 [Error] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\librealsense-exception.h:52 - asic and proj temperatures cannot access the sensor
19:46:05 [Error] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\ds\d400\d400-thermal-monitor.cpp:97 - Error during thermal compensation handling: asic and proj temperatures cannot access the sensor
19:46:16 [Error] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\rs.cpp:237 - [rs2_create_device( info_list:00000207D317E270, index:0 ) UNKNOWN] hr returned: HResult 0x800703e3: "スレッドの終了またはアプリケーションの要求によって、I/O 処理は中止されました。"
19:46:16 [Error] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\rs.cpp:237 - [rs2_delete_device( device:nullptr ) UNKNOWN] null pointer passed for argument "device"
19:46:16 [Warn] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\rs.cpp:373 - null pointer passed for argument "device"
19:46:16 [Warn] C:\jenkins_sys_rsbuild\workspace\LRS_windows_compile_pipeline\src\rs.cpp:3106 - Couldn't refresh devices - hr returned: HResult 0x800703e3: "スレッドの終了またはアプリケーションの要求によって、I/O 処理は中止されました。"
The text was updated successfully, but these errors were encountered: