Contrastive Learning
A self-supervised technique where the model learns by comparing similar (positive) and dissimilar (negative) pairs of examples. It learns representations where similar items are close and different items are far apart.
Why It Matters
Contrastive learning produces excellent embeddings without labeled data. It powers image search, recommendation systems, and multimodal models like CLIP.
Example
Training a model with pairs: the same image with two different crops (positive pair) vs two completely different images (negative pair). The model learns to recognize same-content variations.
Think of it like...
Like a wine tasting where you compare similar wines side by side to notice subtle differences, and then compare with completely different wines to understand major distinctions.
Related Terms
Self-Supervised Learning
A training approach where the model generates its own labels from the data, typically by masking or hiding parts of the input and learning to predict them. No human-annotated labels are needed.
CLIP
Contrastive Language-Image Pre-training — an OpenAI model trained to understand the relationship between images and text. CLIP can match images to text descriptions without being trained on specific image categories.
Embedding
A numerical representation of data (text, images, etc.) as a vector of numbers in a high-dimensional space. Similar items are placed closer together in this space, enabling machines to understand semantic relationships.
Representation Learning
The process of automatically discovering useful features or representations from raw data, rather than manually engineering them. Deep learning excels at learning hierarchical representations.