60What is contrastive / metric learning, and how does it learn good embeddings?▼hardGoogleMetaOpenAI2 replies◆ premiumContrastive learning is how modern embeddings (CLIP, sentence encoders, SimCLR) are actually trained. The signal is the pull-positives-push-negatives objective, the InfoNCE loss, and why the number and hardness of negatives makes or breaks quality.Open full answer →
91You're training embeddings with contrastive/triplet loss. How do you choose pairs, the margin, and negatives?▼hardGoogleMetaCohere2 replies◆ premiumMetric learning lives or dies on the pairs you feed it. Random negatives teach almost nothing, and the margin and mining strategy decide whether the embeddings are any good. Here is how the choices interact.Open full answer →
