Keyword Search vs Vector Search

Two Different Retrieval Models

Keyword search finds documents containing the words in a query. Vector search finds documents semantically similar to the meaning of a query. They are fundamentally different approaches with different strengths and failure modes.

How Each Works

Keyword Search (BM25)

Builds an inverted index — every word maps to the documents containing it. Ranks by term frequency, inverse document frequency, and document length normalization.

Fast, interpretable, no GPU needed. Works on Elasticsearch and OpenSearch.

Vector Search

Embeds query and documents into the same vector space. Finds documents whose vectors are closest to the query vector by cosine similarity or Euclidean distance.

Captures meaning across different word choices. Requires embedding model and vector database.

Where Each Fails

1

Keyword search fails at synonyms, paraphrasing, abbreviations, domain jargon the user doesn't know, and cross-language queries. "Automobile" misses "car."

2

Vector search fails at exact identifier lookup — product codes, names, version numbers, legal clause references. These have no semantic meaning. Also less interpretable: users can't reason about why a result appeared.

3

Vector search is silently wrong when the embedding model is low quality or domain-mismatched. Results look plausible but are incorrect — a dangerous failure mode.

Infrastructure Comparison

Keyword Search

Extremely fast — milliseconds over billions of documents. Mature ecosystem. No GPU required. Low infrastructure complexity.

Vector Search

Requires embedding compute (GPU or API) at index and query time. Needs a vector database. Higher infrastructure complexity and cost per query.

The Practical Takeaway

Neither wins alone

Keyword search excels at exact match and identifier lookup. Vector search excels at natural language queries and cross-vocabulary retrieval. Modern production systems combine both into hybrid search — getting the strengths of each while covering each other's failure modes.

Need Help?

Ask the AI assistant about keyword search, BM25, vector search, or when to use each approach.