Depthmap lighting

[rasmusgo]

Judging the quality of a depth map is hard when it is only colored by the depth because local details are lost. These details can become visible by computing normals and using them for lighting. An alternative to lighting is to simply display the normals by encoding them directly in rgb (as normals maps are typically done). Lighting is more flexible than coloring by normals because it also works together with other ways of coloring the depth data, eg by a colormap, solid color or an image from a color camera.

[emilk]

There is more than one way of computing normals for a depth map though (e.g. look at just the closest four neighbors, or look at a larger neighborhood and ignore outliers, etc etc) so it is unlikely there will be a single solution that fits all.

So allowing users to log a separate normal for each pixel would make sense here. This will be easier once we have:

• Treat tensor values as instances #1756

[haixuanTao]

+1. linking some documentation that does just this:

• https://andreasnaoum.medium.com/visualize-colmaps-image-based-3d-reconstruction-using-rerun-c9987602b279

But I agree that it's not straightforward to make this work.