Set reasonable default values for all the losses and miners etc.

[KevinMusgrave]

The parameters of some losses (e.g. ArcFace) and miners (e.g. MultiSimilarityMiner) don't have default init values, or suggested values in the documentation. This makes it harder for newcomers to use.

[bobiblazeski]

This would really help. Or at least give some examples with any values, starting from blank is the hardest.

[KevinMusgrave]

I'll try to include default values in the next version 0.9.90. Later I might include explanations + maybe images of equations from the papers.

In the meantime, this table has "optimal" hyperparameters that I obtained through experiments on some loss functions: https://kevinmusgrave.github.io/powerful-benchmarker/papers/mlrc/#optimal-hyperparameters. (Side note: the negative pos_margin for ContrastiveLoss is equivalent to 0.)

These hyperparameter were obtained using bayesian optimization. The setup was:

• BN-Inception model pretrained on Imagenet, with a single fully connected layer that outputs 128 dim embeddings
• RMSprop learning rate of 1e-6, and momentum 0.9.
• Batch size 32 unless otherwise stated.

The task was to finetune the pretrained model on 3 datasets, represented by each column: CUB (200 classes of bird images), Cars (196 classes of car images), and SOP (~20000 classes of images off of Ebay).

[bobiblazeski]

Thank you for the quick reply and thank for the work on this wonderful library.

BTW I'm trying to use ArcFace loss and it seems that your constructor doesn't pass the arguments required from the constructor of the base class

https://github.com/KevinMusgrave/pytorch-metric-learning/blob/master/src/pytorch_metric_learning/losses/arcface_loss.py#L11
has none of the required parameters by the LargeMarginSoftmaxLoss

pytorch-metric-learning/src/pytorch_metric_learning/losses/large_margin_softmax_loss.py

Line 15 in ad4b155

def __init__(self, margin, num_classes, embedding_size, scale=1, normalize_weights=False, scale_logits_by_magnitudes=True, **kwargs):

which requires: margin, num_classes, embedding_size

Unless I'm doing something wrong, the ArcFace paper is on my TODO list

[KevinMusgrave]

They should be getting passed via **kwargs. You have to specify those (margin, num_classes, embedding_size) as keyword arguments, which is something I guess I never thought about much because I usually use keyword arguments anyway. I can see how that's really confusing because the documentation makes it look like margin, num_classes, and embedding_size are positional arguments. I could add *args and pass that to the parent class as well. (I just made a separate issue for this: #168 (comment))

[KevinMusgrave]

All the losses and miners have default values in v0.9.90, and equations have been added to many of the losses in the documentation. See the release notes to see breaking changes and new features.