Creating source frames

[jjoachim]

Required Info
Camera Model	D400
Firmware Version
Operating System & Version	Ubuntu 16.04
Kernel Version (Linux Only)
SDK Version	2613547...

There doesn't seem to be a way to make frames myself; I can only grab them from the camera. It'd be nice if there was a way to deep copy and/or create frames from scratch so I can fill them in and use the given API for alignment and pointcloud generation. For instance, I'm interested in processing the RGB image to make a mask image, then align that to the depth frame to create a point cloud. The pointcloud's points would then be labeled by my mask image, and I can do further work from there.

It'd also be nice to be able to create a frameset from two frames to use in the align class api (for reasons stated previously).

Finally, "proccess" is spelled incorrectly various times in rs_processing.hpp (the file name is spelled correctly).

[dorodnic]

Hi @jjoachim
The requirement is completely reasonable, and we are looking into ways to enable this.

At the moment the frame sub-system is trying to:

1. Provide thread-safe API for the user to manage the lifetime of individual frames
2. Ensure fast and (more importantly) deterministic frame allocations when streaming

To accomplish both we have (a somewhat over-complicated) buffer-pool mechanism. That's why there is no API to create one single frame at the moment.

One way to deal with this is to define a custom processing_block:

processing_block pb([&](rs2::frame f, const rs2::frame_source& source)
{
    // source can be used to allocate new frames and send them out
});
frame_queue q;
pb.start(q); // Results from the block will be enqueued into q
pb.invoke(input_frame); // Invoke the lambda above
frame output_frame = q.wait_for_frame(); // Fetch the result

In this case, new frames will be allocated from the buffer-pool that exists inside the processing_block object.
Unfortunately, this API is not well documented yet and isn't very user-friendly either.

In addition, I am currently working on a "software only" device that can be used to generate frames from arbitrary data. It is still a work-in-progress, but will be most likely merged next week. The plan is to use such device to better unit-test point-cloud, sync and align methods, but one could also use it to inject data from an external source. Keeping your use case in mind, I will try to add a decent code-sample alongside this change.

Regarding spelling - the team is diverse and contains non-native English speakers (myself included). Fortunately, none of the misspellings are on the binary API boundary (C API), so we will fix it ASAP.
Thanks for pointing this out.

[jjoachim]

Is there a similar processing block way of creating a frameset from a depth and rgb frame (by copy or ref is fine as long as I know which it is)? I'm digging through the code and found rs2_allocate_composite_frame(...), but there's a lot of indirection I'm having trouble following.

[dorodnic]

Yes, I found an example here: spatial-filter.cpp#85

[jjoachim]

You mentioned merging your "software only" device most likely last week. Did this get merged? If so, could you point me to a code example and I think this issue can be closed.

[dorodnic]

It is currently under review #966 (there is some extra functionality we need to add before it can be merged)

[dorodnic]

Hi @jjoachim
Please review software-device example and measure example and let us know if this helps.

[dorodnic]

Closing this one, feel free to open new issue if you have more questions