losses

All losses are divided into three groups:

  1. classifier-based

  2. pair-based

  3. proxy-based

classifier-based

method

description

CrossEntropyLoss

Cross entropy loss for unsupervised methods

LargeMarginSoftmaxLoss

paper: Large-Margin Softmax Loss for Convolutional Neural Networks

ArcFaceLoss

paper: ArcFace: Additive Angular Margin Loss for Deep Face Recognition

CosFaceLoss

paper: CosFace: Large Margin Cosine Loss for Deep Face Recognition

pair-based

method

description

ContrastiveLoss

paper: Learning a Similarity Metric Discriminatively, with Application to Face Verification

MarginLoss

paper: Sampling Matters in Deep Embedding Learning

TripletLoss

paper: Learning local feature descriptors with triplets and shallow convolutional neural networks

AngularLoss

paper: Deep Metric Learning with Angular Loss

CircleLoss

paper: Circle Loss: A Unified Perspective of Pair Similarity Optimization

FastAPLoss

paper: Deep Metric Learning to Rank

LiftedStructureLoss

paper: Deep Metric Learning via Lifted Structured Feature Embedding

MultiSimilarityLoss

paper: Multi-Similarity Loss With General Pair Weighting for Deep Metric Learning

NPairLoss

paper: Improved Deep Metric Learning with Multi-class N-pair Loss Objective

SignalToNoiseRatioLoss

paper: Signal-To-Noise Ratio: A Robust Distance Metric for Deep Metric Learning

PosPairLoss

paper: Exploring Simple Siamese Representation Learning

proxy-based

method

description

ProxyLoss

paper: No Fuss Distance Metric Learning Using Proxies

ProxyAnchorLoss

paper: Proxy Anchor Loss for Deep Metric Learning

SoftTripleLoss

paper: SoftTriple Loss: Deep Metric Learning Without Triplet Sampling

Example

Take ContrastiveLoss for example:

>>> loss_func = ContrastiveLoss()
>>> x = torch.randn(100, 128)
>>> mat = torch.cdist(x, x)
>>> labels = torch.randint(0, 10, size=(100,))
>>> loss = loss_func(mat, labels.unsqueeze(1), labels.unsqueeze(0))

Class

BaseLoss

class gedml.core.losses.base_loss.BaseLoss(**kwargs)[source]

Bases: gedml.core.modules.with_recorder.WithRecorder

Base loss module. The output of this module will be wrapped with “FINISH” flag which indicates the output doesn’t need to be further processed.

abstract compute_loss(metric_mat, row_labels, col_labels, indices_tuple=None, weights=None, is_same_source=False) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

forward(metric_mat, row_labels, col_labels, indices_tuple=None, weights=None, is_same_source=False) → torch.Tensor[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

CrossEntropyLoss

class gedml.core.losses.classifier_based_loss.cross_entropy_loss.CrossEntropyLoss(**kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

paper: Momentum Contrast for Unsupervised Visual Representation Learning

Cross-entropy loss designed for MoCo.

compute_loss(metric_mat, row_labels, col_labels, is_same_source=False, *args, **kwargs) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

LargeMarginSoftmaxLoss

class gedml.core.losses.classifier_based_loss.large_margin_softmax_loss.LargeMarginSoftmaxLoss(margin=4, scale=1, **kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

modified from: https://github.com/KevinMusgrave/pytorch-metric-learning

paper: Large-Margin Softmax Loss for Convolutional Neural Networks

compute_loss(metric_mat, row_labels, col_labels, indices_tuple=None, is_same_source=False, *args, **kwargs) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

ArcFaceLoss

class gedml.core.losses.classifier_based_loss.arcface_loss.ArcFaceLoss(margin=28.6, scale=64, **kwargs)[source]

Bases: gedml.core.losses.classifier_based_loss.large_margin_softmax_loss.LargeMarginSoftmaxLoss

paper: ArcFace: Additive Angular Margin Loss for Deep Face Recognition

CosFaceLoss

class gedml.core.losses.classifier_based_loss.cosface_loss.CosFaceLoss(margin=0.35, scale=64, **kwargs)[source]

Bases: gedml.core.losses.classifier_based_loss.large_margin_softmax_loss.LargeMarginSoftmaxLoss

paper: CosFace: Large Margin Cosine Loss for Deep Face Recognition

ContrastiveLoss

class gedml.core.losses.pair_based_loss.contrastive_loss.ContrastiveLoss(pos_margin=0, neg_margin=1, **kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

paper: Learning a Similarity Metric Discriminatively, with Application to Face Verification

compute_loss(metric_mat, row_labels, col_labels, indices_tuple, is_same_source=False, *args, **kwargs) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

MarginLoss

class gedml.core.losses.pair_based_loss.margin_loss.MarginLoss(alpha=0.2, beta=1.2, nu=0, num_classes=100, beta_constant=False, is_similarity=False, **kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

paper: Sampling Matters in Deep Embedding Learning

compute_loss(metric_mat, row_labels, col_labels, indices_tuple, is_same_source=False, *args, **kwargs) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

TripletLoss

class gedml.core.losses.pair_based_loss.triplet_loss.TripletLoss(margin=1, **kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

paper: Learning local feature descriptors with triplets and shallow convolutional neural networks

compute_loss(metric_mat, row_labels, col_labels, indices_tuple, is_same_source=False, *args, **kwargs) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

AngularLoss

class gedml.core.losses.pair_based_loss.angular_loss.AngularLoss(alpha=40, **kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

paper: Deep Metric Learning with Angular Loss

compute_loss(metric_mat, row_labels, col_labels, indices_tuple, is_same_source=False, *args, **kwargs) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

CircleLoss

class gedml.core.losses.pair_based_loss.circle_loss.CircleLoss(m=0.4, gamma=80, **kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

modified from: https://github.com/KevinMusgrave/pytorch-metric-learning

paper: Circle Loss: A Unified Perspective of Pair Similarity Optimization

compute_loss(metric_mat, row_labels, col_labels, indices_tuple, weights=None, is_same_source=False) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

FastAPLoss

class gedml.core.losses.pair_based_loss.fast_ap_loss.FastAPLoss(num_bins=10, **kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

modified from: https://github.com/KevinMusgrave/pytorch-metric-learning

paper: Deep Metric Learning to Rank

compute_loss(metric_mat, row_labels, col_labels, indices_tuple=None, weights=None, is_same_source=False) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

LiftedStructureLoss

class gedml.core.losses.pair_based_loss.lifted_structure_loss.LiftedStructureLoss(neg_margin=1, pos_margin=0, **kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

paper: Deep Metric Learning via Lifted Structured Feature Embedding

compute_loss(metric_mat, row_labels, col_labels, indices_tuple=None, weights=None, is_same_source=False) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

MultiSimilarityLoss

class gedml.core.losses.pair_based_loss.multi_similarity_loss.MultiSimilarityLoss(alpha=2, beta=50, base=0.5, **kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

paper: Multi-Similarity Loss With General Pair Weighting for Deep Metric Learning

compute_loss(metric_mat, row_labels, col_labels, indices_tuple=None, weights=None, is_same_source=False) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

NPairLoss

class gedml.core.losses.pair_based_loss.n_pair_loss.NPairLoss(**kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

Work with NPairSelector (Recommend)

paper: Improved deep metric learning with multi-class n-pair loss objective

compute_loss(metric_mat, row_labels, col_labels, indices_tuple=None, weights=None, is_same_source=False) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

SignalToNoiseRatioLoss

class gedml.core.losses.pair_based_loss.signal_to_noise_ratio_loss.SignalToNoiseRatioLoss(**kwargs)[source]

Bases: gedml.core.losses.pair_based_loss.contrastive_loss.ContrastiveLoss

paper: Signal-To-Noise Ratio: A Robust Distance Metric for Deep Metric Learning

PosPairLoss

class gedml.core.losses.pair_based_loss.pos_pair_loss.PosPairLoss(**kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

Designed for SimSiam.

paper: Exploring Simple Siamese Representation Learning

compute_loss(metric_mat, row_labels, col_labels, indices_tuple, weights=None, is_same_source=False) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

ProxyLoss

class gedml.core.losses.proxy_based_loss.proxy_loss.ProxyLoss(gamma, **kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

paper: No Fuss Distance Metric Learning Using Proxies

compute_loss(metric_mat, row_labels, col_labels, is_same_source=False, *args, **kwargs) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

ProxyAnchorLoss

class gedml.core.losses.proxy_based_loss.proxy_anchor_loss.ProxyAnchorLoss(margin=0.1, alpha=32, **kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

paper: Proxy Anchor Loss for Deep Metric Learning

compute_loss(metric_mat, row_labels, col_labels, indices_tuple=None, weights=None, is_same_source=False) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor

SoftTripleLoss

class gedml.core.losses.proxy_based_loss.soft_triple_loss.SoftTripleLoss(centers_per_class=10, num_classes=100, la=20, gamma=0.1, margin=0.01, **kwargs)[source]

Bases: gedml.core.losses.base_loss.BaseLoss

paper: SoftTriple Loss: Deep Metric Learning Without Triplet Sampling

compute_loss(metric_mat, row_labels, col_labels, indices_tuple=None, weights=None, is_same_source=False) → torch.Tensor[source]

Compute loss value.

Parameters

  • metric_mat (torch.Tensor) – Metric matrix.

  • row_labels (torch.Tensor) – Labels of matrix rows.

  • col_labels (torch.Tensor) – Labels of matrix columns.

  • indices_tuple (dict) – Dict that has two keys: “tuples” and “flags”

  • weights (torch.Tensor) – Can be element-wised, tuple-wised etc.

  • is_same_source (bool) –

Returns

Final loss value (a tensor value).

Return type

torch.Tensor