ROSBAG: Single Frame Capture? [C++]

[HippoEug]

Required Info
Camera Model	D435
Firmware Version	05.09.02.00
Operating System & Version	Win 10
Kernel Version (Linux Only)
Platform	PC
SDK Version	2.16.1
Language	C++
Segment	others

Introduction

Guten tag, I am trying to capture only 1 frame of the depth frame and save in .bag format to do further offline calculations. Is this possible?

Issue

To achieve this, I looked at rs-record-playback.cpp and tried to replicate it. Here is a snippet of my code to capture as little as data as possible...

Capturing Data

pipe->stop(); // Stop the pipeline with default configuration
rs2::config cfg; // Declare a new configuration
cfg.enable_record_to_file("test.bag");

pipe->start(cfg); // Open file with configuration
// Some delay here?
pipe->stop(); // Stop the pipeline that holds the file and the recorder

pipe = std::make_shared<rs2::pipeline>(); //Reset the shared pointer with a new pipeline
pipe->start(); // Resume streaming with default configuration

Results

From the code snippet above, with a pipe->start() followed by an immediate pipe->stop(), this generated a 27KB BAG file as seen on File Explorer.

Any attempts to open this on Intel RealSense Viewer.exe crashes the whole application.. I guess this file is too small?

Any suggestions on modifications to be made to capture 1 frame?

Danke..! 😄

[philipjames44]

I replied on the other thread, but for documentation's sake: https://github.com/IntelRealSense/librealsense/blob/master/doc/stepbystep/getting_started_with_openCV.md

This can get you frame by frame using OpenCV and display it with ImShow. If you just want 1 frame you can use the wait key to wait for a few milliseconds, and save the file to a png or jpeg from the Mat file. If you want ROS I'm personally not versed in it, but I don't see that it would be a big issue to just toss the image into a ros bag.

[HippoEug]

Thanks @philipjames44 , I will take a look at the example you shared. Good day!

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment]
Hi HippoEug,

Do you still need help on this one? We are going to close this if there is nothing else.

Thanks!

[HippoEug]

I will try the example provided. Thank you @RealSense-Customer-Engineering

[HippoEug]

I found a workaround, but this method compromises the color camera frames recorded to the .bag file since the color camera takes more time to initialize (settle down).

The first thing you want to do is to record a .bag file as per normal, but record as little as possible. To achieve this, I did std::this_thread::sleep_for(std::chrono::milliseconds(1000)); in between pipe->start(cfg); and pipe->stop();.

A more complete code when capturing .bag modified from rs-capture-playback.cpp:

if (ImGui::Button("record", { 50, 50 }))
{
    // If it is the start of a new recording (device is not a recorder yet)
    if (!device.as<rs2::recorder>())
    {
        pipe->stop(); // Stop the pipeline with the default configuration
        pipe = std::make_shared<rs2::pipeline>();
        rs2::config cfg; // Declare a new configuration
        cfg.enable_record_to_file("a.bag");
        pipe->start(cfg); //File will be opened at this point
        device = pipe->get_active_profile().get_device();

		// Edited Start
		std::this_thread::sleep_for(std::chrono::milliseconds(1500));
		pipe->stop(); // Stop the pipeline that holds the file and the recorder
		pipe = std::make_shared<rs2::pipeline>(); //Reset the shared pointer with a new pipeline
		pipe->start(); // Resume streaming with default configuration
		device = pipe->get_active_profile().get_device();
		recorded = true; // Now we can run the file
		recording = false;
		// Edited End
    }
}

To convert these files into a single frame capture, either to .png or .csv, I modified rs-convert.cpp. Since I am primarily interested in capturing a single frames to decrease conversion/processing time, I can ignore many frames and just dive right to convert and save a frame before jumping out of the conversion loop.

Here is a more complete code modified from rs-convert.cpp.

while (true) {
	auto frameset = pipe->wait_for_frames();
		
	int posP = static_cast<int>(playback.get_position() * 100. / duration.count()); // Current position probably

	if (posP > progress) { // If current position > previous progress, update to new progress %
		progress = posP;
		//std::cout << posP << "%" << "\r" << std::flush;
		std::cout << posP << "%" << std::endl;
	}

	std::cout << "frameset[0].get_frame_number() = " << frameset[0].get_frame_number() << std::endl;
	std::cout << "frameNumber = " << frameNumber << std::endl;

	if (frameset[0].get_frame_number() == 5 || frameset[0].get_frame_number() == 6 || frameset[0].get_frame_number() == 7) { // If end of frame, bag loops again, hence get_frame_number resets to 1 and is smaller
		std::for_each(converters.begin(), converters.end(), [&frameset](std::shared_ptr<rs2::tools::converter::converter_base>& converter) {
			converter->convert(frameset);
		});

		std::for_each(converters.begin(), converters.end(), [](std::shared_ptr<rs2::tools::converter::converter_base>& converter) {
			converter->wait();
		});

		break;
	}

	frameNumber = frameset[0].get_frame_number();
}

From the above example, I captured either frames 5, 6 or 7 in case the convert program skips frames, which happens quite often.

[dorodnic]

Hi @HippoEug
Following-up on this (and couple of others) issue, @lramati added rs_export.hpp. The way it works, is you include rs_export.hpp, create save_single_frameset and pass the frameset you got from wait_for_frames to it (something like frameset.apply_filter(single_saver).
This will create new bag file for every frameset you pass to it, and save only frames in the frameset into ROS-bag

[HippoEug]

Hi @dorodnic , that is awesome. I will give it a shot and let you know how it goes :)

Which release will this go into?

[dorodnic]

Hi @HippoEug
Sorry for the delay, it is already part of 2.17.1 and 2.18.0

[HippoEug]

Thank you very much for the update @dorodnic

[soarwing52]

I tried the save_single_frameset in python, since I also requires one frame date per capture
I also tried sleep(0.01) & sleep(0.02)

however, the captured frames change depends on the exposure time, even I had

    color_sensor = profile.get_device().query_sensors()[1]
    color_sensor.set_option(rs.option.auto_exposure_priority, False)

and the save single frame function is as my issue in #4020