Embeddings for PMs: What 'Semantic Similarity' Actually Buys You

An embedding is a list of numbers that positions a piece of text in a "meaning space," where things with similar meaning end up near each other. Semantic similarity search finds nearby points instead of matching exact words. That's why it retrieves "chargeback" when you search "refund" — and why it can also miss things a keyword search would have caught instantly.

Quick Answer: Embeddings turn text into coordinates in a high-dimensional space where distance approximates meaning. This buys you retrieval that survives paraphrasing and synonyms, but it costs you precision on exact identifiers, negations, and rare or technical terms — which is exactly why most production systems pair embeddings with keyword search rather than relying on either alone.

You don't need to understand the linear algebra to make good product decisions here. You do need a working mental model of what embeddings are actually doing, where that model breaks down in front of real users, and what to build instead of, or alongside, pure semantic search. This article builds that model with one running example, then sets up the case for hybrid retrieval as a follow-on decision.

What Is an Embedding, in Plain Product Terms?

An embedding is a fixed-length vector — typically 384 to 3,072 numbers — that an embedding model produces for a piece of text, capturing something like its meaning as a point in space. Two pieces of text that mean similar things get mapped to points that sit close together; two pieces of text that mean different things land far apart. Similarity search is just measuring that distance.

Think of it as a map, not a filing cabinet. A keyword index is a filing cabinet: it stores exact words and looks them up by matching characters. An embedding index is more like a map of a city, where "similar neighborhoods" — similar meanings — sit near each other geographically, even if the street names share zero letters.

How the Model Learns "Nearness"

The model isn't reasoning about meaning the way a person does. It learns from patterns of co-occurrence and context across enormous amounts of text — words and phrases that show up in similar surrounding contexts get pulled toward similar coordinates during training.

This is the same intuition behind word2vec, one of the earliest widely-used embedding techniques, which researchers at Google demonstrated in 2013 could capture surprisingly rich relationships — the famous example being that the vector arithmetic "king − man + woman" landed close to "queen." Modern sentence and document embedding models (from providers like OpenAI, Cohere, and open-source options built on transformer architectures) extend that same core idea to full sentences and paragraphs, not just single words.

Why Distance, Not Keywords, Is the Point

• Paraphrase tolerance: a user who types "how do I get my money back" retrieves a policy titled "Refund Eligibility" even though no words overlap.
• Synonym handling: "cancel my plan" and "terminate my subscription" land near each other despite sharing almost no vocabulary.
• Cross-lingual potential: some multilingual embedding models place a sentence and its translation close together, since the meaning — not the surface tokens — is what's being encoded.
• Robustness to typos and phrasing variance: minor rewording barely moves the point, unlike exact-match keyword search, which fails on the smallest mismatch.

That's the brilliant part. Now for the part that breaks.

The Failure Mode: Where Semantic Nearness Misleads You

Embeddings fail predictably on three shapes of query: exact identifiers, negations, and rare or domain-specific terms. Each failure comes from the same root cause — the model is trained to cluster by general topical or contextual similarity, not by the precise logical relationship your product actually needs.

Once you can name these three failure shapes, you can predict, before you ship, which query types will disappoint users — instead of discovering it in a support ticket.

Exact Identifiers Get Blurred Into Their Neighborhood

Order numbers, SKUs, ticket IDs, and account numbers are exactly the kind of string embeddings handle worst. A model trained on natural language sees "ORD-88213" and treats it structurally like other alphanumeric codes nearby in its training distribution — not as a unique key that must match exactly or not at all.

Search for order number ORD-88213 in a pure semantic index and you may get back ORD-88214, ORD-77213, or a support article that merely mentions order numbers in passing — all scored as "similar" because they share surface structure and context, not because any of them are the record you need.

Exact-match lookups are a keyword search problem, not a meaning problem. Embeddings were never the right tool for this job — the fix isn't a better embedding model, it's routing identifier-shaped queries to exact or fuzzy string match instead.

Negations Barely Move the Vector

"I want a refund" and "I do not want a refund" are, embedding-wise, almost neighbors. The word "not" is a small token in a long sentence, and most embedding models weight overall topical content far more heavily than the single logical operator that flips the sentence's meaning.

This is a well-documented weakness across dense retrieval and broader NLP research; papers studying negation understanding in transformer-based models — including work presented at ACL and EMNLP venues over the past several years — have repeatedly found that semantic similarity models struggle to represent negation as a meaning-inverting operation rather than a minor lexical addition. For a support or knowledge tool, that means a query like "orders that were not refunded" can retrieve documents about orders that were refunded, ranked confidently high.

Rare and Technical Terms Cluster Badly

Embedding models learn their geometry from training data, and terms that appeared rarely — or in inconsistent contexts — end up with noisier, less reliable positions. A jargon-heavy internal term, a newly coined feature name, or a niche technical acronym may sit awkwardly close to unrelated concepts simply because the model saw too few examples to place it precisely.

This matters most for B2B and enterprise products, where your users' vocabulary is dense with terms your embedding model never saw enough of during training — internal SKUs, compliance codes, proprietary feature names. The rarer and more specific the term, the less you should trust the embedding's placement of it.

The Running Example: "Refund" vs. "Chargeback" vs. an Order Number

Consider three real queries a support search box will actually receive, and how a pure embedding search handles each:

The plain-language takeaway: embeddings are excellent at "these are about the same topic" and unreliable at "these mean the same specific thing." A refund and a chargeback are topically adjacent but operationally different — merging them in search results can send a user down the wrong support flow entirely.

Why This Isn't a Bug to Patch — It's a Retrieval Architecture Decision

Semantic-only retrieval fails on identifiers, negations, and rare terms because of what embeddings fundamentally optimize for, not because of a fixable implementation mistake. No amount of prompt tuning on top of the retrieved results fixes a document that never made it into the retrieved set in the first place.

This is the core insight product teams miss: retrieval quality is decided before generation, at the chunking and indexing stage — a decision explored in depth in chunking strategy and retrieval quality. If the wrong document gets retrieved, no amount of clever prompting downstream will recover the right answer.

The Product Question Hiding Inside "Semantic Similarity"

Every embedding-based search silently answers a question your product team never explicitly decided: which sense of this query do we want retrieved? "Refund" could mean the refund policy, a specific transaction, or the broader dispute process — and the embedding model picks one interpretation implicitly, based on training statistics, not your product's intent.

This is one of the four product decisions RAG forces teams to make explicitly instead of leaving to default model behavior — and it's a decision most teams don't realize they're avoiding until a user complains that search "returned the wrong thing" for a query that felt obviously right to them.

Where Hybrid Retrieval Comes In

The fix that most production RAG systems converge on is hybrid retrieval: run a keyword/lexical search (commonly BM25, a decades-old ranking function still used because it's excellent at exact and rare-term matches) alongside semantic vector search, then merge or re-rank the results.

• Keyword search catches what embeddings miss: exact IDs, rare technical terms, and negation-sensitive phrasing where literal word presence matters.
• Semantic search catches what keyword search misses: paraphrases, synonyms, and conceptually related content with no lexical overlap.
• A re-ranking step (often a smaller, more precise model applied only to the combined candidate set) resolves conflicts between the two result sets before anything reaches the user or the LLM.

This isn't a fringe pattern — it's close to the default architecture recommended across the RAG literature and by major vector-database and search vendors once teams move past a prototype. The tradeoff — and when hybrid is worth the added complexity versus fine-tuning your embedding model instead — is its own decision, covered in RAG vs. fine-tuning: a decision framework.

How to Recognize the Failure Mode Before Users Do

You can predict most embedding failures without running a single query, by checking your product's query patterns against three risk signals: identifier density, negation frequency, and vocabulary rarity.

Before shipping any semantic search feature, audit your actual (or expected) query logs against these signals rather than testing only with clean, well-formed example queries.

A Pre-Launch Risk Checklist

1. Count identifier-shaped queries. Do users search by order number, ticket ID, SKU, or account number? If yes, exact-match retrieval is mandatory, not optional.
2. Sample negation-bearing queries. Look for "not," "isn't," "without," "except," and similar constructs in real or anticipated search logs.
3. List your rare vocabulary. Internal product names, compliance terms, and niche jargon are the terms most likely to be poorly placed by a general-purpose embedding model.
4. Test synonym pairs deliberately. Confirm the "topically adjacent but operationally distinct" trap — like refund vs. chargeback — doesn't merge concepts your support team treats very differently.
5. Decide your merge strategy up front. Will keyword and semantic results be combined by a simple score blend, a reciprocal-rank fusion formula, or a dedicated re-ranker model?

Making the Retrieval Sense Explicit: The Prodinja Connection

The hardest part of embedding-based retrieval isn't the math — it's deciding, in plain product language, which meaning of a query you actually want surfaced. That's a product-judgment question, not a machine-learning one, and it's easy to skip past when a vector database "just works" in a demo.

Key Takeaways

• Embeddings encode meaning as position: nearby vectors mean similar meaning, which is why semantic search survives paraphrasing and synonyms that break keyword search.
• Exact identifiers are the clearest failure case: order numbers, SKUs, and ticket IDs need exact or fuzzy string match, not semantic similarity.
• Negation is quietly dangerous: "not" and similar operators barely shift an embedding's position, so negated queries can retrieve the opposite of what a user meant.
• Rare and technical terms cluster unreliably: the less training data a term had, the less you should trust the embedding's placement of it.
• Topically adjacent isn't operationally identical: refund and chargeback sit close in vector space but require different handling — don't let proximity substitute for your actual taxonomy.
• Hybrid retrieval (keyword plus semantic, with re-ranking) is the standard production fix, not a fine-tuned embedding model alone.
• "Which meaning do we want retrieved" is a product decision hiding inside every embedding search — naming it explicitly, before launch, prevents most of these failures from reaching users.