Infrared image overexposed in sunlight (D435)

[scott-robotics]

Hi

I've been using the D435 outdoors and I cannot get correct infrared camera exposure in bright sunlight. Using both auto-exposure, and manual minimum exposure and gain, results in overexposure (see images)

Any way to increase shutter speed, or decrease gain below 16?

Thanks!

Sunny:

Overcast:

Info
Camera Model	D435
Firmware Version	5.10.13
Operating System & Version	Ubuntu 16.04
Kernel Version (Linux Only)	4.15.0-32-generic
Platform	PC
SDK Version	2.16.5

[MartyG-RealSense]

Intel's advice about using the camera in strong light can be found in the link below. Click on the image thumbnail in the post to view the advice in full size.

https://forums.intel.com/s/question/0D70P0000069GjOSAU/noise-in-realsense-d435-pointcloud?language=en_US

[RiccMazz]

Hi Scottnothing,
you are not alone!
As far as I am aware of, there is no documented way to set shutter speed and gain beyond the limits you experienced.
We have tried hard to use RS D435 outdoor and had to come to the conclusion that the only RealSense usable in sunlight is RS D415. We came to the conclusion that D435 was, maybe, designed to fly drones in full darkness!
The only possibility we found to use it in sunlight is to use a IR filter but, again, if you have a choice go for D415.

[MartyG-RealSense]

D435 has been used outdoors successfully on a drone in sunlight. The discussion linked to below has an attached short video of the user's drone flight.

The main issue that was experienced was small artifacts on the image. I suggested reducing exposure to around 1 ms. Unfortunately, the user did not reply to report whether doing so had solved their artifact issue.

https://forums.intel.com/s/question/0D50P0000490XM0SAM/noise-in-realsense-d435-pointcloud

Shutter speed can be affected indirectly by changing the color exposure, as shutter speed is also known as 'exposure time'. This is the approach that some RealSense users have taken to achieving an effect similar to a physical shutter speed alteration.

[scott-robotics]

Thanks for the responses. I have tried both auto-exposure and manually setting expose time to 1ms. Both produce something like the images I posted. The drone video you linked has only RGB stream, it would be interesting to see the infra stream of the same scene. I am concerned only with the infra stream. I notice also that overexposure happens on light-colored surfaces like concrete or sand. Grass doesn't seem to be an issue.

[MartyG-RealSense]

Someone did a drone flight with a D435 and Jetson TX2. The video has a depth and IR stream.

https://www.youtube.com/watch?v=5tY2A-_VBi8

They did an accompanying blog entry about the flight.

https://mikeisted.wordpress.com/2018/04/22/first-flight-intel-realsense-d435-on-jetson-tx2/

[RiccMazz]

Gentlemen,
as scottnothing is trying to say, his problem has no relationship with flying drones or insects.
Just try to look at a pavement in a sunny day, as he posted, and you will get wide areas of undetermined pixels due to saturated areas in the infrared cameras. IMHO, having hands-on experience, D435 is unusable in direct sunlight unless filtered.

[agrunnet]

This may help a small amount. There is a new featured that has been enabled that allows for "fractional exposure". This means you can now set the exposure to less than 20. As low as 1. In principle this allows for 1/16 exposure, and it has been used to make the D435 work outdoors in bright sunlight. Just make sure you have FW that is after 5.10.0.

Here is an example image of lighting above my office chair. (It was getting dark outside so could not check it there.)

So first check the manual exposure to see if this mitigates your problem.
The fractional exposure feature is now also part of the autoexposure algorithm.
However, you may have to set the "AE ROI" so it autogains on the bright ground.

Finally, you are absolutely right that another solution is to add an ND filter (optical attenuator) to the front to attenuate always.

[agrunnet]

Regarding optical filters, I particularly like these guys. Very responsive and good prices. You can get ND filters, as well as more wavelength selective attenuation.
http://www.astraproducts.com/index.asp

Others are
http://www.leefilters.com/

But make sure to get a spec for the full spectrum. Many times they only show their spectral curves out to 750nm. You want to attenuate across the full spectrum of sensitivity upto about 1000nm.
BTW, an OD of 1 means 10x. An OD of 2 means 100x.

[agrunnet]

Another good option to consider is thin-film linear polarizers. Use two on top of each other and rotate one and you will be able to create your own variable attenuator. These can be quite cheap.

[scott-robotics]

Thanks very much for the advice, and particularly for info on filters, @agrunnet. I actually haven't seen enough sun recently to double check setting exposure to 1 vs 20. At the time, rqt_reconfigure definitely showed that exposure was set to 1, but I cannot confirm what the camera thinks its exposure setting was.

[agrunnet]

Yes Astra Products is where we got the variable ND filters/IR-pass filters.