Saving pyrealsense2.pyrealsense2.depth_frame object

[negvet]

Required Info
Camera Model	D435i
Firmware Version	05.12.01.00
Operating System & Version	Linux (Ubuntu16)
Kernel Version (Linux Only)	(e.g. 4.14.13)
Platform	PC
SDK Version	{ 2.33.1.1388 }
Language	{python }
Segment	{Robot }

Issue Description

It is not possible to save pyrealsense2.pyrealsense2.depth_frame/color_frame/composite_frame in memory.

I iterate over a bag file and want to save all frames. Within the pipeline, I can process the frames the way I want, but I also want to save them for postprocessing. I do not want to save images, point clouds or anything else. I need to save pyrealsense2.pyrealsense2.depth_frame object itself.

It turns out that I can append a list or NumPy array with pyrealsense2.pyrealsense2.depth_frame object only 32 times. Below is my implementation

d435_filename = '435.bag'

# Setup:
cfg = rs.config()
cfg.enable_device_from_file(d435_filename)
cfg.enable_stream(rs.stream.depth, 848, 480, rs.format.z16, 30)
pipe = rs.pipeline()
profile = pipe.start(cfg)

d435_data_list = []
first_timestamp = None
counter = 0
while(True):
    frames = pipe.wait_for_frames()
    depth_frame = frames.get_depth_frame()
    
    if depth_frame.get_timestamp() == first_timestamp:
        print('reached the first frame - reading bag file completed')
        print('timestamp',depth_frame.get_timestamp())
        break
    if first_timestamp is None:
        first_timestamp = depth_frame.get_timestamp()
        print('first_timestamp',first_timestamp)
    
    print('current-first',depth_frame.get_timestamp()-first_timestamp)
    print('frame #',counter)
    counter +=1
    
    d435_data_list.append(depth_frame)                                  # -> Doesn't Work
#   d435_data_list.append(np.asanyarray(depth_frame.get_data()).copy()) # -> Works

pipe.stop()

Here is the output:

first_timestamp 1584948362869.346
current-first 0.0
frame # 0
current-first 32.331787109375
frame # 1
current-first 65.487548828125
frame # 2
current-first 98.315673828125
...
current-first 1198.07080078125
frame # 30
current-first 1231.48291015625
frame # 31
current-first -48.611083984375
frame # 32
reached the first frame - reading bag file completed
timestamp 1584948362869.346

But actually the bag file contains much more than 32 frames. I am able to go through all of them if I comment d435_data_list.append(depth_frame) line. I am also not able to copy.copy(depth_frame) or copy.deepcopy(depth_frame) - some issue with pickle.

Question:
How to save pyrealsense2.pyrealsense2.depth_frame/color_frame/composite_frame objects in memory for postprocessing?

Thanks!

Best
Evgeny

[MartyG-RealSense]

Hi Evgeny,

I recalled a recent case in which a RealSense user who was using a Python array could not process more than 15 frames. The research in that case led me to the comment in the link below regarding saving frames inside a collection.

#946 (comment)

In regard to saving frames in memory, I wonder if you might be able to do so with the Keep() function and then apply post-processing on the frames all at the same time as a batch operation once the pipeline is closed. The links below have examples of Python code for Keep():

#3164 (comment)

#3121 (comment)

[negvet]

@MartyG-RealSense

Keep() indeed allows storing the frames in memory.

Thanks a lot!

[MartyG-RealSense]

I'm very glad you were able to solve your problem with Keep(). Thanks for the update!