BERT
Bidirectional Encoder Representations from Transformers — a language model developed by Google that reads text in both directions simultaneously. BERT excels at understanding language rather than generating it.
Why It Matters
BERT revolutionized search engines and NLP tasks like question answering and sentiment analysis. Google uses BERT to better understand search queries.
Example
Google Search using BERT to understand that in 'parking on a hill with no curb,' the word 'no' is critical and changes the entire meaning of the query.
Think of it like...
Like reading a mystery novel where you already know the ending — understanding the full context helps you interpret every clue more accurately.
Related Terms
Transformer
A neural network architecture introduced in 2017 that uses self-attention mechanisms to process sequential data in parallel rather than sequentially. Transformers are the foundation of modern LLMs like GPT, Claude, and Gemini.
Masked Language Model
A training approach where random tokens in the input are replaced with a special [MASK] token and the model learns to predict the original tokens from context. This is how BERT was pre-trained.
Natural Language Processing
The branch of AI that deals with the interaction between computers and human language. NLP enables machines to read, understand, generate, and make sense of human language in a useful way.
Pre-training
The initial phase of training a model on a large, general-purpose dataset before specializing it for specific tasks. Pre-training gives the model broad knowledge and capabilities.