[question] Comparison between different versions

[xzhub]

Is there any results comparison between different Meshroom versions? I couldn't find any in youtube or any other sources. According to my tests with limited test sets, I feel the 2019 version is better than v2020 and v2021.
It might because of the poor images I fed to Meshroom, but the 2019 version give me a better result with the same parameters(sift+akaze, high FeatureExtraction describerPreset, guided matching) and same images.

v2019:

v2021:

I'd like to see more, and in depth comparision, if there is any. Could any one provide that info? I think the dev team must have some testsets. is there any blog post or videos talking about this?

[fabiencastan]

We are doing intensive tests to avoid regressions. If you find one exemple with a repeatable regression that would be interesting to analyse.

Could you try to run it multiple times? The process is not repeatable, so 2 different runs could end-up in different results with some variation in quality. So the difference may come from variations between runs instead of changes between software version.

One problem is that the notion of density on sift feature extraction has changed a bit, so that's difficult to make side-by-side comparison.

Could you try without akaze?
Could you try with the new dsp-sift?

[xzhub]

Should I try to replace sift with dspsift, or just simply add dspsift?

Should I still use akaze If I work with a project of face scan? (in previous version, akaze is recommanded for human skin, so I am wondering if this has been changed)

[fabiencastan]

If you use DSP-SIFT, use it instead of SIFT (there is no interest in using them together).

Multiple changes have been done on SIFT. So it would be good to re-evaluate if it still make sense to combine it with AKAZE. SIFT or DSP-SIFT may be enough now.

[xzhub]

DSP-SIFT only works great on this project! It's better than before definitely. Will try SIFT only as well.

[xzhub]

SIFT only is not as good as DSP-SIFT only. But still better than SIFT+AKAZE

[fabiencastan]

Thanks for sharing your results.
I end up with the same conclusion in my various tests too. I'm considering changing the default value to dsp-sift in the 3D reconstruction pipeline (for the panorama use case, SIFT is still a bit better).

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue is closed due to inactivity. Feel free to re-open if new information is available.