An embedding is a numerical representation of meaning. When you embed a piece of text, you convert it into a list of numbers — called a vector — where the position of that vector in mathematical space encodes semantic content.
Similar meanings produce similar vectors. "The car broke down" and "The vehicle stopped working" produce vectors that are close together in embedding space, even though they share no words.
Computers operate on numbers. Natural language does not come in numeric form. Embeddings solve this by assigning a position in high-dimensional space to every piece of content, where that position encodes meaning.
Before embeddings, machines matched text by finding exact words in common — keyword matching. With embeddings, machines can find content that means the same thing even when phrased completely differently. This unlocks semantic search, recommendation systems, and retrieval-augmented generation.
Semantic proximity — Content with similar meanings produces vectors that are close together. "Happy" and "joyful" are near each other; "Happy" and "database" are far apart.
Generalization — The same concept expressed in different words produces similar vectors. The embedding captures meaning, not phrasing.
Fixed dimensionality — Regardless of input length, the model always outputs a vector of the same size (e.g., 1536 numbers). This consistency allows all vectors to be compared in the same space.
Embedding models are neural networks — typically transformer-based — trained to place semantically similar content close together and semantically different content far apart. During training, the model processes billions of text examples and learns what "similar meaning" looks like across many contexts.
OpenAI text-embedding-3-small and text-embedding-3-large, Sentence-BERT (SBERT), Cohere Embed, and Google's text-embedding models. Each has different tradeoffs in dimensionality, speed, cost, and domain quality.
The choice of embedding model matters. Models trained on general text perform differently than models fine-tuned for code, scientific papers, or multilingual content. Always match your embedding model to your use case.
Ask the AI assistant about embeddings, vectors, or how embedding models work.