Using python to access sensor_timestamp from metadata

[XupengZhu-SW]

Required Info
Camera Model	D400
Firmware Version
Operating System & Version	Linux (Ubuntu 22)
Kernel Version (Linux Only)	5.19.0-50-generic
Platform	PC
SDK Version	pyrealsense2
Language	python3.8
Segment	Robot

Hi there, I am trying to access accurate time stamp when the image is captured by the sensor. For previous post I know that there are 4 stamps availabe:
frames.get_frame_metadata(rs.frame_metadata_value.backend_timestamp)
frames.get_frame_metadata(rs.frame_metadata_value.frame_timestamp)
frames.get_frame_metadata(rs.frame_metadata_value.sensor_timestamp)
frames.get_frame_metadata(rs.frame_metadata_value.time_of_arrival)

From the explanations, the sensor_timestamp is what I need. However, I cannot access this attribute using pyrealsense2 package. I got error "RuntimeError: metadata not available".

I also tried build from source with "cmake . -DENFORCE_METADATA=true -DBUILD_PYTHON_BINDINGS:bool=true -DPYTHON_EXECUTABLE=/home/...", but the build do not build a python binding.

Any idea to access time_stamp using python? Thanks in advance for the help!

Best,
XP

[MartyG-RealSense]

Hi @XupengZhu-SW Did you apply a patch script to the kernel before building the librealsense SDK, please? Doing so is necessary when building from source code in order to enable support for hardware timestamps.

If you are using a kernel older than 5.13 then the ./scripts/patch-realsense-ubuntu-lts.sh patch script should be used. If the kernel is 5.13 or 5.15 then use ./scripts/patch-realsense-ubuntu-lts-hwe.sh

[XupengZhu-SW]

Hi Marty,

Thanks for the quick response!

I find my linux kernel version is 5.19.0-50-generic. The script ./scripts/patch-realsense-ubuntu-lts-hwe.sh does not support my kernel. Do you know which patch script should I use?

Moreover, I didn't notice the readme for building from source.

Thanks,
XP

[MartyG-RealSense]

Kernel 5.19 is not yet supported in the RealSense SDK as of the current version 2.54.1. If you require hardware metadata then you can bypass the kernel and not need to use a patch script by building the SDK from source code with CMake with the build flag -DFORCE_RSUSB_BACKEND=TRUE included in the CMake build instruction. An RSUSB build of the SDK includes hardware metadata support.

[XupengZhu-SW]

Hi Marty,

Thank you for your quick reply! I tried building with the flag -DFORCE_RSUSB_BACKEND=TRUE, but there is a bug.

I can access the sensor_timestamp now, but the reading is weird:

frames = self.pipeline.wait_for_frames()
dt1 = frames.get_frame_metadata(rs.frame_metadata_value.backend_timestamp) / 1e3
dt2 = frames.get_frame_metadata(rs.frame_metadata_value.frame_timestamp) / 1e3
dt3 = frames.get_frame_metadata(rs.frame_metadata_value.time_of_arrival) / 1e3
dt4 = frames.get_frame_metadata(rs.frame_metadata_value.sensor_timestamp) / 1e3
print(dt1, dt2, dt3, dt4)

I got prints:

0.0 328418.977 1690911559.39 328409.143

These readings are in unit of seconds and should have similar values, i.e., around 1690911559.39.

FYI, I copy-past the built pyrealsense2 (pybackend2.cpython-38-x86_64-linux-gnu.so
pybackend2.cpython-38-x86_64-linux-gnu.so.2
pybackend2.cpython-38-x86_64-linux-gnu.so.2.54.1
pyrealsense2.cpython-38-x86_64-linux-gnu.so
pyrealsense2.cpython-38-x86_64-linux-gnu.so.2.54
pyrealsense2.cpython-38-x86_64-linux-gnu.so.2.54.1)
from build/wrapper/python to my python lib path.

Best,
XP

[MartyG-RealSense]

A RealSense user at #9891 took the approach of dividing the sensor timestamp by 1000./1000. instead of 1e3. An example of a timestamp that this instruction returned was 888.8259390000001

f"Sensor: {frames.get_frame_metadata(rs.frame_metadata_value.sensor_timestamp)/1000./1000.}"

[XupengZhu-SW]

Unfortunately dividing the sensor timestamp by "1000./1000." get unreasonable results as well. Build from source in a 5.19 kernel may be problematic.

[MartyG-RealSense]

What is the result like if you simply remove the division factor?

dt4 = frames.get_frame_metadata(rs.frame_metadata_value.sensor_timestamp)

[XupengZhu-SW]

I print out all the time stamps as follows:

frames = self.pipeline.wait_for_frames()
time_stamp = frames.get_frame_metadata(rs.frame_metadata_value.backend_timestamp)
t = time.time()
t1 = frames.get_frame_metadata(rs.frame_metadata_value.backend_timestamp)
t2 = frames.get_frame_metadata(rs.frame_metadata_value.frame_timestamp)
t3 = frames.get_frame_metadata(rs.frame_metadata_value.time_of_arrival)
t4 = frames.get_frame_metadata(rs.frame_metadata_value.sensor_timestamp)
t5 = frames.get_timestamp()
print(t, t1, t2, t3, t4, t5)

I got:

1690995514.0715308 0 9461724 1690995514047 9449002 1690995514034.6802

Ideally, all the time stamps should be close to the current time, i.e., "1690995514" in second, but only ".time_of_arrival" and ".get_timestamp()" are reasonable.

[MartyG-RealSense]

Different timestamps have different characteristics. At #2188 (comment) a RealSense team member provides a list of timestamp types and descriptions for them, and further information about how the timestamps work at #4525