Losses

MoCo

eztorch.losses.compute_moco_loss(q, k, k_global, use_keys, queue, temp=0.2, rank=0)[source]

Compute the SCE loss.

Parameters:

  • q (Tensor) – The representations of the queries.

  • k (Tensor) – The representations of the keys.

  • k_global (Tensor) – The global representations of the keys.

  • use_keys (bool) – Whether to use the non-positive elements from key.

  • temp (float, optional) – Temperature applied to the query similarities.

    Default: 0.2

  • rank (int, optional) – Rank of the device for positive labels.

    Default: 0

Return type:

Tensor

Returns:

The loss.

eztorch.losses.compute_mocov3_loss(q, k, temp=1.0, rank=0)[source]

Compute the MoCov3 loss.

Parameters:

  • q (Tensor) – The representations of the queries.

  • k (Tensor) – The global representations of the keys.

  • temp (float, optional) – Temperature for softmax.

    Default: 1.0

  • rank (int, optional) – Rank of the device for positive labels.

    Default: 0

Return type:

Tensor

Returns:

The loss.

ReSSL

eztorch.losses.compute_ressl_loss(q, k, k_global, use_keys, queue, mask, temp=0.1, temp_m=0.04, LARGE_NUM=1000000000.0)[source]

Compute the RESSL loss.

Parameters:

  • q (Tensor) – The representations of the queries.

  • k (Tensor) – The representations of the keys.

  • k_global (Tensor) – The global representations of the keys.

  • use_keys (bool) – Whether to use the non-positive elements from key.

  • queue (Tensor) – The queue of representations.

  • mask (Tensor | None) – Mask of positives for the query.

  • temp (float, optional) – Temperature applied to the query similarities.

    Default: 0.1

  • temp_m (float, optional) – Temperature applied to the keys similarities.

    Default: 0.04

  • LARGE_NUM (float, optional) – Large number to mask elements.

    Default: 1000000000.0

Return type:

Tensor

Returns:

The loss.

eztorch.losses.compute_ressl_mask(batch_size, num_negatives, use_keys=True, rank=0, world_size=1, device='cpu')[source]

Precompute the mask for ReSSL.

Parameters:

  • batch_size (int) – The local batch size.

  • num_negatives (int) – The number of negatives besides the non-positive key elements.

  • use_keys (bool, optional) – Whether to use the non-positive elements from the key as negatives.

    Default: True

  • rank (int, optional) – Rank of the current process.

    Default: 0

  • world_size (int, optional) – Number of processes that perform training.

    Default: 1

  • device (Any, optional) – Device that performs training.

    Default: 'cpu'

Return type:

Tensor | None

Returns:

The mask.

SCE

eztorch.losses.compute_sce_loss(q, k, k_global, use_keys, queue, mask, coeff, temp=0.1, temp_m=0.07, LARGE_NUM=1000000000.0)[source]

Compute the SCE loss.

Parameters:

  • q (Tensor) – The representations of the queries.

  • k (Tensor) – The representations of the keys.

  • k_global (Tensor) – The global representations of the keys.

  • use_keys (bool) – Whether to use the non-positive elements from key.

  • queue (Tensor) – The queue of representations.

  • mask (Tensor) – Mask of positives for the query.

  • coeff (Tensor) – Coefficient between the contrastive and relational aspects.

  • temp (float, optional) – Temperature applied to the query similarities.

    Default: 0.1

  • temp_m (float, optional) – Temperature applied to the keys similarities.

    Default: 0.07

  • LARGE_NUM (float, optional) – Large number to mask elements.

    Default: 1000000000.0

Return type:

Tensor

Returns:

The loss.

eztorch.losses.compute_sce_mask(batch_size, num_negatives, use_keys=True, rank=0, world_size=1, device='cpu')[source]

Precompute the mask for SCE.

Parameters:

  • batch_size (int) – The local batch size.

  • num_negatives (int) – The number of negatives besides the non-positive key elements.

  • use_keys (bool, optional) – Whether to use the non-positive elements from the key as negatives.

    Default: True

  • rank (int, optional) – Rank of the current process.

    Default: 0

  • world_size (int, optional) – Number of processes that perform training.

    Default: 1

  • device (Any, optional) – Device that performs training.

    Default: 'cpu'

Return type:

Tensor

Returns:

The mask.

SCE tokens

eztorch.losses.compute_sce_token_loss(q, k, k_global, queue, mask_sim_q, mask_sim_k, mask_prob_q, mask_log_q, coeff, temp=0.1, temp_m=0.07, LARGE_NUM=1000000000.0)[source]

Compute the SCE loss for several tokens as output.

Parameters:

  • q (Tensor) – The representations of the queries.

  • k (Tensor) – The representations of the keys.

  • k_global (Tensor) – The global representations of the keys.

  • queue (Tensor | None) – The queue of representations.

  • mask_sim_q (Tensor | None) – Mask to ignore similarities for the query.

  • mask_sim_k (Tensor) – Mask to ignore similarities for the keys.

  • mask_prob_q (Tensor) – Mask of positives for the query.

  • mask_log_q (Tensor | None) – Mask of elements to keep after applying log to the query distribution.

  • coeff (Tensor) – Coefficient between the contrastive and relational aspects.

  • temp (float, optional) – Temperature applied to the query similarities.

    Default: 0.1

  • temp_m (float, optional) – Temperature applied to the keys similarities.

    Default: 0.07

  • LARGE_NUM (float, optional) – Large number to mask elements.

    Default: 1000000000.0

Return type:

Tensor

Returns:

The loss.

eztorch.losses.compute_sce_token_masks(batch_size, num_tokens, num_negatives, positive_radius=0, keep_aligned_positive=True, use_keys=True, use_all_keys=False, rank=0, world_size=1, device='cpu')[source]

Precompute the masks for SCE with tokens.

Parameters:

  • batch_size (int) – The local batch size.

  • num_tokens (int) – The number of tokens per instance.

  • num_negatives (int) – The number of negatives besides the non-positive key elements.

  • positive_radius (int, optional) – The radius of adjacent tokens to consider as positives.

    Default: 0

  • keep_aligned_positive (bool, optional) – Whether to keep the aligned token as positive.

    Default: True

  • use_keys (bool, optional) – Whether to use the non-positive elements from the aligned key as negatives.

    Default: True

  • use_all_keys (bool, optional) – Whether to use the non-positive elements from all the gathered keys as negatives.

    Default: False

  • rank (int, optional) – Rank of the current process.

    Default: 0

  • world_size (int, optional) – Number of processes that perform training.

    Default: 1

  • device (Any, optional) – Device that performs training.

    Default: 'cpu'

Return type:

Tuple[Tensor, Tensor, Tensor, Tensor, Tensor]

Returns:

  • Mask to ignore similarities for the query.

  • Mask to ignore similarities for the keys.

  • Mask of positives for the query.

  • Mask to keep log values.

  • The number of positives for the specific token.

SimCLR

eztorch.losses.compute_simclr_loss(z, z_global, pos_mask, neg_mask, temp)[source]

Compute the simCLR loss.

Parameters:

  • z (Tensor) – The local representations.

  • z_global (Tensor) – The gathered representations.

  • pos_mask – Positives mask.

  • neg_mask – Negative masks.

  • temp (float) – Temperature for softmax.

Return type:

Tensor

Returns:

The loss.

eztorch.losses.compute_simclr_masks(batch_size, num_crops=2, rank=0, world_size=1, device='cpu')[source]

Compute positive and negative masks for SimCLR.

Parameters:

  • batch_size (int) – The local batch size per iteration.

  • num_crops (int, optional) – The number of crops per instance.

    Default: 2

  • rank (int, optional) – Rank of the current process.

    Default: 0

  • world_size (int, optional) – Number of processes that perform training.

    Default: 1

  • device (device, optional) – Device that performs training.

    Default: 'cpu'

Return type:

tuple[Tensor, Tensor]

Returns:

The positive and negative masks.

Spot loss

eztorch.losses.compute_spot_loss_fn(class_preds, class_target, has_label, ignore_class, class_weights=None, mixup_weights=None, class_loss_type=LossType.BCE, alpha=0.25, gamma=2)[source]

Compute the soccernet loss function which is a classification loss.

Parameters:

  • class_preds (Tensor) – Predictions for the classes. Expected shape: (B, T, C).

  • class_target (Tensor) – Multi-label encoded class. Can be continuous label in case of mixup. Expected shape: (B, T, C).

  • has_label (Tensor) – Whether there is a label. Expected shape: (B’, T, C).

  • ignore_class (Tensor) – Whether to ignore class. Expected shape: (B’, T, C).

  • class_weights (Optional[Tensor], optional) – Weights of negatives and positives for BCE loss. Expected shape: (2, C).

    Default: None

  • mixup_weights (Optional[Tensor], optional) – Weights for mixup for loss. Expected shape: (B’, T, C).

    Default: None

  • class_loss_type (LossType, optional) – Type of loss to use. Can be BCE, softmax or Focal.

    Default: LossType.BCE

  • alpha (float, optional) – For focal loss.

    Default: 0.25

  • gamma (float, optional) – For focal loss.

    Default: 2

Return type:

Tensor

Returns:

The reduced sum of the classification losses.