Alternating Optimization for Color Map Optimization

[chunduri11]

IMPORTANT: Please use the following template to report the bug.

---

Describe the bug

Alternating Optimization for Color Map Optimization is specified in the paper as an option which can be scalable and parallalizable to speed up the algorithm.
I could not see any reference to Alternating Optimization any where in the source code(c++) for Color map Optimization.
Is the Alternating Optimization for Color Map Optimization implemented as part of the code for CMO or not?

When I use the CMO for my own sample it is very slow(runs on cpu).
Do you have any suggestion to speed up the algorithm and implement it in gpu(CUDA)?

THANKS

[sanskar107]

Alternating Optimization is not implemented currently. Would you like to contribute with our help?
Also, we are working on shifting entire reconstruction module on newly added Tensor library which will be accelerated using CUDA. We will be rolling them out in future releases. Stay tuned.

[chunduri11]

Thanks for the quick reply. Very encouraging.

I will be happy to contribute. Currently I am trying to understand the Color map optimisation from the cpp code in the o3d repo and the paper. Soon after that I will try to work on Alternating Optimisation code.

The CMO paper says that the Alternating Optimisation is scalable and optimisation steps(two steps T and C) can be parallelised, with out much compromise on the quality of the result.
Do you have any idea how much speed up can result with Alternating Optimisation, w.r.t non-rigid CMO?

Regarding the future releases with reconstruction module on Tensor library, do you know how soon this is expected?

Running CMO using existing Open3d api is extremely slow especially the non-rigid version, I am very much looking forward to these changes.

[chunduri11]

@sanskar107 Can you please help me with above questions.
And how do I proceed with alternating optimization, I have not contributed to Open3D before.

[theNded]

There might have been misunderstandings. It has been implemented according to the paper.

Parallelized alternating optimization for rigid optimization is here
https://github.com/intel-isl/Open3D/blob/master/cpp/open3d/pipelines/color_map/ColorMapOptimization.cpp#L160 for equation (14) and
https://github.com/intel-isl/Open3D/blob/master/cpp/open3d/pipelines/color_map/ColorMapOptimization.cpp#L203 for the closed form solution (13).

For non-rigid here are the references:
https://github.com/intel-isl/Open3D/blob/master/cpp/open3d/pipelines/color_map/ColorMapOptimization.cpp#L95 for pose estimation and
https://github.com/intel-isl/Open3D/blob/master/cpp/open3d/pipelines/color_map/ColorMapOptimization.cpp#L135 for warping update.

Color map optimization is slow since it is running with high-quality images and very dense correspondences. A potential upgrade will be using the CUDA backend, which is non-trivial and will be on our roadmap. But this is improving widgets such as correspondence matching and residual estimation, rather than implementing alternating optimization which is already implemented and well-established.