Extrinsics not available in recorded bag

[cso-mark]

---

Required Info
Camera Model	D435i
Firmware Version	05.13.00.50
Operating System & Version	Windows 10
Kernel Version (Linux Only)	N/A
Platform	Windows x86
SDK Version	2.50.0.3812
Language	python
Segment	Robot

Issue Description

I recorded a bag using the Realsense Viewer and loaded it from python using pyrealsense2. I got an error when trying to access the extrinsics between the color and depth sensors. I used .get_extrinsics_to() which threw an exception with message Requested extrinsics are not available!

You can replicate with the following script.

import pyrealsense2 as rs
import os.path
import argparse

parser = argparse.ArgumentParser(description="No intrinsics bug")
# Add argument which takes path to a bag file as an input
parser.add_argument("-i", "--input", type=str, help="Path to the bag file")

args = parser.parse_args()
ctx = rs.context()
playback = ctx.load_device(filename=args.input)
sensors = playback.query_sensors()
profiles = []
for s in sensors:
    print("sensor ", s)
    for p in s.get_stream_profiles():
        profiles.append(p.as_video_stream_profile())
        print("\tprofile ", p, p.as_video_stream_profile().get_intrinsics())
print(profiles)
try:
    profiles[0].get_extrinsics_to(profiles[1])
except RuntimeError as e:
    print("extrinsics from ", profiles[0], " to ", profiles[1], " failed with ", e)
try:
    profiles[1].get_extrinsics_to(profiles[0])
except RuntimeError as e:
    print("extrinsics from ", profiles[1], " to ", profiles[0], " failed with ", e)                

The output on my machine is

python extrinsics_fail.py -i 20220427_155611.bag
sensor  <pyrealsense2.sensor: "Stereo Module">
        profile  <pyrealsense2.[video_]stream_profile: Depth(0) 848x480 @ 30fps Z16> [ 848x480  p[425.454 235.173]  f[424.823 424.823]  Brown Conrady [0 0 0 0 0] ]
sensor  <pyrealsense2.sensor: "RGB Camera">
        profile  <pyrealsense2.[video_]stream_profile: Color(0) 640x480 @ 30fps RGB8> [ 640x480  p[322.448 249.45]  f[617.775 617.948]  Inverse Brown Conrady [0 0 0 0 0] ]
[<pyrealsense2.video_stream_profile: Depth(0) 848x480 @ 30fps Z16>, <pyrealsense2.video_stream_profile: Color(0) 640x480 @ 30fps RGB8>]
extrinsics from  <pyrealsense2.video_stream_profile: Depth(0) 848x480 @ 30fps Z16>  to  <pyrealsense2.video_stream_profile: Color(0) 640x480 @ 30fps RGB8>  failed with  Requested extrinsics are not available!
extrinsics from  <pyrealsense2.video_stream_profile: Color(0) 640x480 @ 30fps RGB8>  to  <pyrealsense2.video_stream_profile: Depth(0) 848x480 @ 30fps Z16>  failed with  Requested extrinsics are not available!

[MartyG-RealSense]

Hi @cso-mark In #1887 a RealSense user who was reading from a bag file took the approach of defining profiles for color and depth and then accessing those profiles in the get_extrinsics_to instruction.

This is how they obtained the depth to color extrinsics.

depth_intrin = depth_frame.profile.as_video_stream_profile().intrinsics
color_intrin = color_frame.profile.as_video_stream_profile().intrinsics
depth_to_color_extrin = depth_frame.profile.get_extrinsics_to(color_frame.profile)

Presumably the color to depth extrinsics would therefore be accessed like this:

color_to_depth_extrin = color_frame.profile.get_extrinsics_to(depth_frame.profile)

[cso-mark]

Reading through the linked issue, the author reports that that script never worked. In the corrected script after that, the call to get_extrinsics_to does not appear. Futhermore, I think my code is pretty much the same as the one you linked on a conceptual level. I retrieve the depth and color profile objects and call get_extrinsics_to.

Is this possibly a bug in the SDK that the extrinsics are not properly recorded into the bag?

[MartyG-RealSense]

When a bag file is used as the camera data source by a script, it should behave like when a live camera is providing the data. Typically, this changing of data sources from live camera to bag is done by inserting the SDK instruction cfg.enable_device_from_file(<filename>) before the pipe.start() line. I note that you are not using this instruction in your script.

The Python script in the link below provides an example of the instruction's use.

https://stackoverflow.com/questions/58482414/frame-didnt-arrived-within-5000-while-reading-bag-file-pyrealsense2

[MartyG-RealSense]

Hi @cso-mark Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.