Memory Issue Python, Possible Memory Leak?

[yeric1789]

Required Info
Camera Model	D435
Firmware Version	5.12.09.00
Operating System & Version	Windows 10 Version 10.0.19042 Build 19042
Platform	x86 AMD
SDK Version	2.40
Language	python 3.7.8

Hi,

I have been working on a multi camera set up for intel realsense cameras and have run into an issue where the memory usage increases drastically while recording for extended periods of time. This only occurs when the following code is run when collecting the frames.

for (serial, device) in enabled_devs_in.items(): indx, frameset = device.pipeline.try_wait_for_frames(3) #frameset will be a pyrealsense2.composite_frame object if frameset.size() > 0: frames[serial] = thresh_filter.process(frameset.first_or_default(rs.stream.depth))

I have also tried this

frames = {} while len(frames) < len(enabled_devs_in.items()): for (serial, device) in enabled_devs_in.items(): streams = device.pipeline_profile.get_streams() frameset = device.pipeline.poll_for_frames() if frameset.size() == len(streams): frames[serial] = {} for stream in streams: key_ = stream.stream_type() frames[serial][key_] = thresh_filter.process(frameset.first_or_default(key_)) return frames

As well as this

while len(frames) < len(enabled_devs_in.items()): for (serial, device) in enabled_devs_in.items(): frameset = device.pipeline.poll_for_frames() if frameset.size() > 0: frames[serial] = thresh_filter.process(frameset.first_or_default(rs.stream.depth))
Please see the screen shots below. (Please note that the CPU usage is high due to the simultaneous compression, I have checked multiple times if the compression is the issue but if you note the second process which is taking up most of the CPU usage you will see that it is indeed the collection process which is eating up memory).
BEFORE EARLY IN RECORDING

**AFTER PERIOD OF TIME **

I believe I have narrowed down this behavior to have something to do with the device.pipeline.poll_for_frames()/ .try_for_frames / wait_for_frames() methods as this behavior does not occur without these methods. I have also ruled out personal configurations with numpy as well as configurations relating to the compression and storage of the frames.

Note that this behavior does scale up and has continued to eat more and more memory when I am recording for long periods of time, and since this will be in use for 24 hour recording periods I am in need of a solution.
I am unsure if this has something to do with the documentation https://intelrealsense.github.io/librealsense/python_docs/_generated/pyrealsense2.pipeline.html?highlight=wait_for_frames#pyrealsense2.pipeline.wait_for_frames "However, if the application maintains too long history, the device may lack memory resources to produce new frames" where it states something about memory resources yet it is fuzzy in terms of which "device" it is referring to (either the PC or the cameras).

Any help anyone could provide on this issue would be much appreciated. Thank you

[MartyG-RealSense]

Hi @yeric1789 The situation that you described has similarities to a recent Python case on Windows in which duplicate frames were apparently being generated for every frame. The RealSense user in that case was also using try_wait_for_frames. They suspected that this section of their code was the cause of the problem, and confirmed that the problem still occurred when using wait_for_frames instead. The problem did not occur for them when using the earlier 2.35 librealsense SDK version.

#8949 (comment)

I tested their frame counting script and was able to replicate the reported problem. After discussing the issue with Intel, I subsequently filed an official bug report with Intel on 7 May 2021.

[yeric1789]

Hi @MartyG-RealSense,

Thanks for the reply. Would downgrading the SDK version also mess with the newer version firmware I have on the intel realsense cameras themselves? Could you possibly guide me uninstalling the newer SDK version and reinstalling the older librealsense SDK version along with pyrealsense as I am unsure exactly how I would go about doing that. Thanks

[MartyG-RealSense]

As you are using Windows, installing an older SDK version should be as simple as downloading and running the automated installer file for the SDK from the Assets file list at the bottom of the link below. This is because when the SDK is installed with this installer file, it replaces the previously installed SDK version.

https://github.com/IntelRealSense/librealsense/releases/tag/v2.35.0

The firmware could be a complicating factor. Firmware version 5.12.8.200 and newer require librealsense 2.39.0 or newer to be used, due to internal firmware changes. So it may be necessary to downgrade to firmware 5.12.7.100 to have compatibility with 2.35.0. I would recommend trying 2.35.0 with your currently installed recent firmware first to determine whether a downgrade is going to be necessary.

As your computer is an x86, it should be compatible with installing pyrealsense2 using the pip install pyrealsense2 method instead of building it from source code.

I recently installed Python on a Windows laptop. The process that I followed was:

1. Go to the Microsoft Store and search for python in the store to find a download page for it, and install Python on the computer.

I recommend using Python 3.7, as support for pip install pyrealsense2 on Python 3.8 and 3.9 was only added to the SDK recently.

2. Open the Windows Command Prompt interface. You can find it by typing command into the text box at the bottom of the Windows screen, next to the button in the corner where the PC is shut down.

You can also find a link to the Microsoft Store as a pop-up window within Windows by typing microsoft store into the text box.

3. Type pip into the Command Prompt to first verify that Pip is installed and working. Then input pip install pyrealsense2 to install the Python wrapper.

[yeric1789]

Hi @MartyG-RealSense,

When I run intel realsense viewer I get the following error

Invalid Value in rs2_get_option(options:000002C682870B90, option:Emitter On Off):
hwmon command 0x7d failed.
Error type: No data to return (-21).

[MartyG-RealSense]

Do you still have firmware 5.12.9.0 installed in the camera, please?

[yeric1789]

I don't know how would I check and uninstall it.

[MartyG-RealSense]

The Info option at the top of the Viewer side-panel will display the camera details, including the currently installed firmware version.

In regard to updating the firmware, you could first obtain the 5.12.7.100 firmware as a bin file from the firmware Releases page.

https://dev.intelrealsense.com/docs/firmware-releases

And then in the Viewer, go to the More option at the top of the options side-panel, select the Update Firmware menu option and navigate to the location on your computer where you saved the firmware bin file and select it to commence the firmware update.

[yeric1789]

Hi @MartyG-RealSense,

I was able to get it working with python 3.9 and I am not seeing any errors nor is the memory increasing in the same way I was seeing before. Thanks for all your help.

[MartyG-RealSense]

That's great news @yeric1789 :) You are very welcome!