Issues opening depth + infrared on the F200

[chungs31]

Hi there,

I am trying to get the example files to work with the RS F200, but it seems to me that the depth and infrared streams look strange when they are open simultaneously (on v4l). I've attached some pictures and would appreciate if any input on why this is going on.

Firmware: 2.60
OS: (regrettably) 15.04, linux 4.4.
chipset: Intel 7 series/C216 family.

[sgorsten]

It looks like you might not have applied our uvcvideo.ko kernel patches, as described here:
https://github.com/IntelRealSense/librealsense/blob/master/doc/installation.md#ubuntu-1404-lts-installation

The F200 multiplexes its infrared and depth images on the same UVC device, using a number of simple but nonetheless proprietary formats. Video4Linux cannot handle these formats without some slight modifications. However, due to a quirk of how Video4Linux works, an attempt to set the video mode to an "unknown" format will fail, but leave the device in whatever video mode it would default to. For the F200, this means that an unpatched system will always return 640x480 16-bit depth images at 60 fps, hence the "correct" behavior when you request only depth.

Please let us know whether or not you've applied the kernel patches, and if not, whether or not the problem persists after you do so. Thank you for your time and patience.

[chungs31]

Thanks for the reply. I distinctly remember patching the kernel, as that one took the longest time out of the others.

I can try redoing the steps, but I do remember updating the kernel and spending a bit of time shallow cloning the repo.

[teknotus]

This quickest way to be sure your kernel is patched is to run this with the camera plugged in.

v4l2-ctl --list-formats --device=/dev/videoX

Where X is the device node for the depth camera. With the F200 you get only one named format if it isn't patched, and a bunch if it is.

I'm pretty sure you got the right format though. The way to get depth, and infrared at the same time on the F200 is with a muxed 24 bit format. The first 16 bits are depth, and the last 8 are infrared. In both the depth, and IR images there is banding which tells me you got the depth data. There are also shadows that aren't from the projector getting blocked which tells me you got the IR data.

I think the unpacking routine is moving 32 bits at a time instead of 24. It's taking me a while to grasp how it's supposed to work with the templates. One of the scripts installs a very new version of gcc so maybe this is one of the issues where a c++11 implementation bug breaks things. You could try using your own unpacking routine using fewer language features so that it's easier to debug. One byte array in, and two out. Once you have the array of two byte values filled cast it to 16 bit.

Something like...

for(int i = 0 ; i < input_array ; i += 3){
  depth_char[i] = input_array[i];
  depth_char[i+1] = input_array[i+1];
  ir[i] = input_array[i+2];
}
uint16_t * depth = (uint16_t *)depth_char;

This is completely untested code hereby released into the public domain with no warranty. It may also be used with the license of this project.

[sgorsten]

As teknotus noted above, running:

v4l2-ctl --list-formats --device=/dev/videoX

should yield a list of formats including at least the FourCCs 'INVZ', 'INVI', and 'INZI' if the kernel patch was applied and loaded correctly. Please let us know if you see these formats when enumerating

Note: If the F200 is the only video device plugged into your machine, the depth/infrared device should be /dev/video1, while the color device should be /dev/video0.

[chungs31]

Interestingly, this is what I get from v4l2.

schung@penguin:~$ v4l2-ctl --list-formats --device=/dev/video1
ioctl: VIDIOC_ENUM_FMT
    Index       : 0
    Type        : Video Capture
    Pixel Format: 'YUYV'
    Name        : YUYV 4:2:2

    Index       : 1
    Type        : Video Capture
    Pixel Format: ''
    Name        : 5a564e49-2d90-4a58-920b-773f1f2

    Index       : 2
    Type        : Video Capture
    Pixel Format: ''
    Name        : 495a4e49-1a66-a242-9065-d01814a

    Index       : 3
    Type        : Video Capture
    Pixel Format: ''
    Name        : 52564e49-2d90-4a58-920b-773f1f2

    Index       : 4
    Type        : Video Capture
    Pixel Format: ''
    Name        : 49524e49-2d90-4a58-920b-773f1f2

    Index       : 5
    Type        : Video Capture
    Pixel Format: ''
    Name        : 49564e49-57db-5e49-8e3f-f479532

    Index       : 6
    Type        : Video Capture
    Pixel Format: ''
    Name        : 494c4552-1314-f943-a75a-ee6bbf0

schung@penguin:~$ v4l2-ctl --list-formats --device=/dev/video0
ioctl: VIDIOC_ENUM_FMT
    Index       : 0
    Type        : Video Capture
    Pixel Format: 'YUYV'
    Name        : YUYV 4:2:2

I think the unpacking routine is moving 32 bits at a time instead of 24.

I thought it would be this too, but now I am not sure if the kernel was patched properly. I'll do it again and update.

[chungs31]

Seems to work after removing the repo and doing it again. Not sure where it went wrong but thanks!

[sgorsten]

Sorry for the inconvenience.