How does rs-record-playback.cpp work? [C++]

[HippoEug]

Required Info
Camera Model	D400
Firmware Version	05.09.02.00
Operating System & Version	Win 10
Kernel Version (Linux Only)
Platform	PC
SDK Version	2.16.1
Language	C++
Segment	others

Introduction

Hello community! 😺 My goal of this question is to record just 1 frame and save it as a .bag (rosbag) format if possible. If this is not possible, this should not be a problem as well.

My reason for wanting only 1 frame in a .bag file is for performance, to decrease processing time required when using rs-convert.cpp.

I have asked this similar question once here. I tried the given suggestion, but it does not seem to work. My question is how does the API determine when to start the recording of .rosbag file in the code? I will elaborate further below after Goals.

Goal

Using rs-record-playback.cpp, I managed to capture a .bag file, which I then threw it into rs-convert.cpp to convert into a .csv file with each individual frame (x, y, z) coordinates.

My theory is the smaller the .bag file, the lesser frames it contains which means the shorter the time it needs for rs-convert.cpp to convert the .bag file to .csv file(s).

rs-convert

File Explorer with ~0.5s of recording, theoretically should have 15 frames converted, but 9 actual frames

Issue

The code to do a recording to .bag file seems pretty straight forward. However unlike rs-capture.cpp etc which does:

rs2::frameset data = pipe.wait_for_frames(); // Wait for next set of frames from the camera
rs2::frame depth = color_map.process(data.get_depth_frame()); // Find and colorize the depth data

I do not find any code in rs-record-playback.cpp that does any function which manages the frames. All it does is create a config cfg with the function cfg.enable_record_to_file().

My own code segment to record ~0.5s of .bag file

if (ImGui::Button("record", { 50, 50 }))
{
    // If it is the start of a new recording (device is not a recorder yet)
    if (!device.as<rs2::recorder>())
    {
        pipe->stop(); // Stop the pipeline with the default configuration
        pipe = std::make_shared<rs2::pipeline>();
        rs2::config cfg; // Declare a new configuration
        cfg.enable_record_to_file("a.bag");
        pipe->start(cfg); //File will be opened at this point
        device = pipe->get_active_profile().get_device();

        // Edited Start
        std::this_thread::sleep_for(std::chrono::milliseconds(500));
        pipe->stop(); // Stop the pipeline that holds the file and the recorder
        pipe = std::make_shared<rs2::pipeline>(); //Reset the shared pointer with a new pipeline
        pipe->start(); // Resume streaming with default configuration
        device = pipe->get_active_profile().get_device();
        recorded = true; // Now we can run the file
        recording = false;
        // Edited End
    }
}

Based on the above code segment, pipe->start(cfg) starts the recording and pipe->stop() stops the recording, and no where to specify how many frames I would like to record. Is this correct? I tried looking into cfg.enable_record, but all I can find is this:

void enable_record_to_file(const std::string& file_name)
{
       rs2_error* e = nullptr;
       rs2_config_enable_record_to_file(_config.get(), file_name.c_str(), &e);
       error::handle(e);
}

Edit

Alternatively, I can do it in the rs-convert.cpp code to just convert 1 frame, correct?

[HippoEug]

NOTE: For readability's sake, I rewrote my comment from another thread

I found a workaround, but this method compromises the color camera frames recorded to the .bag file since the color camera takes more time to initialize (settle down).

The first thing you want to do is to record a .bag file as per normal, but record as little as possible. To achieve this, I did std::this_thread::sleep_for(std::chrono::milliseconds(1000)); in between pipe->start(cfg); and pipe->stop();.

A more complete code when capturing .bag modified from rs-capture-playback.cpp:

if (ImGui::Button("record", { 50, 50 }))
{
    // If it is the start of a new recording (device is not a recorder yet)
    if (!device.as<rs2::recorder>())
    {
        pipe->stop(); // Stop the pipeline with the default configuration
        pipe = std::make_shared<rs2::pipeline>();
        rs2::config cfg; // Declare a new configuration
        cfg.enable_record_to_file("a.bag");
        pipe->start(cfg); //File will be opened at this point
        device = pipe->get_active_profile().get_device();

		// Edited Start
		std::this_thread::sleep_for(std::chrono::milliseconds(1500));
		pipe->stop(); // Stop the pipeline that holds the file and the recorder
		pipe = std::make_shared<rs2::pipeline>(); //Reset the shared pointer with a new pipeline
		pipe->start(); // Resume streaming with default configuration
		device = pipe->get_active_profile().get_device();
		recorded = true; // Now we can run the file
		recording = false;
		// Edited End
    }
}

To convert these files into a single frame capture, either to .png or .csv, I modified rs-convert.cpp. Since I am primarily interested in capturing a single frames to decrease conversion/processing time, I can ignore many frames and just dive right to convert and save a frame before jumping out of the conversion loop.

Here is a more complete code modified from rs-convert.cpp.

while (true) {
	auto frameset = pipe->wait_for_frames();
		
	int posP = static_cast<int>(playback.get_position() * 100. / duration.count()); // Current position probably

	if (posP > progress) { // If current position > previous progress, update to new progress %
		progress = posP;
		//std::cout << posP << "%" << "\r" << std::flush;
		std::cout << posP << "%" << std::endl;
	}

	std::cout << "frameset[0].get_frame_number() = " << frameset[0].get_frame_number() << std::endl;
	std::cout << "frameNumber = " << frameNumber << std::endl;

	if (frameset[0].get_frame_number() == 5 || frameset[0].get_frame_number() == 6 || frameset[0].get_frame_number() == 7) { // If end of frame, bag loops again, hence get_frame_number resets to 1 and is smaller
		std::for_each(converters.begin(), converters.end(), [&frameset](std::shared_ptr<rs2::tools::converter::converter_base>& converter) {
			converter->convert(frameset);
		});

		std::for_each(converters.begin(), converters.end(), [](std::shared_ptr<rs2::tools::converter::converter_base>& converter) {
			converter->wait();
		});

		break;
	}

	frameNumber = frameset[0].get_frame_number();
}

From the above example, I captured either frames 5, 6 or 7 in case the convert program skips frames, which happens quite often.

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment]
Hi HippoEug,

Is the workaround you found good enough for your case?

Thanks!

[RealSenseCustomerSupport]

Hi HippoEug,

If there is noting else, this one will be closed soon.

Thanks!