Mixup Technique is helpfull?

[abdikaiym01]

Hi, have you tested the technique mixup (Mixup: Beyond Empirical Risk Minimization) for the different datasets? Is with this technique models can show better performance on dataset IJBC?

[leondgarse]

I only tried the vanilla mixup augment method very earlier, and the results not good. I think it's not compatible with margin loss functions like ArcFace:

• Mixup will expect a ground truth prediction like, let's say 0.5 mixup, [0, 0, ..., 0.5, ..., 0.5, 0, ...]
• ArcFace makes a margin for truth predictions, like 0.8 -> 0.4, 0.5 -> 0.1. This makes them pretty low.

Some basic results using EfficientNetV2B0 + MS1MV3:

• ArcFace: lfw: 0.9975, cfp_fp: 0.975714, agedb_30: 0.976333
• ArcFace + Mixup: lfw: 0.997833, cfp_fp: 0.973571, agedb_30: 0.974333