Embeddings for PMs: How AI Turns Meaning Into Coordinates

Embeddings are a technique that converts text, images, or other content into a list of numbers — a vector — placed in a shared mathematical space, so that items with similar meaning land near each other. That's the whole trick behind semantic search, retrieval-augmented generation, and deduplicating messy feature requests: meaning, expressed as coordinates.

Quick answer: An embedding is a list of numbers (a vector) that represents the meaning of a piece of text or image. Similar meanings produce nearby numbers, so measuring the distance between two vectors is how AI systems find "related" content — the basis of semantic search, RAG, and duplicate detection.

What Are Embeddings? Meaning as Coordinates

An embedding is a fixed-length list of numbers — often hundreds or thousands of them — that a model generates to represent a piece of content's meaning. Two pieces of text with related meaning get placed close together in this numerical space, even if they share zero words. Distance becomes a proxy for similarity.

Think of it like latitude and longitude for concepts instead of places. Just as two cities close on a map tend to share a climate, two pieces of text close in vector space tend to share a topic, intent, or sentiment. The map has far more than two dimensions — modern models from OpenAI use 1,536 or 3,072 numbers per embedding — but the geographic intuition holds.

A few properties make this useful for product work:

• Distance = similarity. The standard metric is cosine similarity, which measures the angle between two vectors rather than their raw magnitude — two vectors pointing the same direction score close to 1, unrelated ones score near 0.
• No shared vocabulary required. "Cancel my plan" and "I want to stop my subscription" can sit near each other even though they share only one word.
• Any content type can be embedded. Text, images, audio, and even user-behavior sequences can all be turned into vectors, which is why embeddings underpin recommendation engines as well as search.

Here's a simplified illustration of how a handful of short phrases might cluster once embedded — real vectors have hundreds of dimensions, but two are enough to show the pattern:

That clustering is not keyword matching — it's a numerical model of meaning. Traditional keyword search would miss most of these pairings because it looks for shared substrings, not shared intent.

The King-Queen Analogy, Carefully

The classic illustration of embeddings is an arithmetic trick: take the vector for king, subtract man, add woman, and the closest resulting vector is queen. It's a genuinely elegant demonstration that these coordinate spaces encode relationships, not just categories — gender, in this case, becomes a consistent direction you can add or subtract.

This example comes from Tomas Mikolov and colleagues at Google, who introduced the word2vec method in 2013 and showed that simple vector arithmetic could recover analogies like this one. Stanford's GloVe project (Pennington, Socher, and Manning, 2014) built on the same idea, training embeddings on word co-occurrence statistics across huge text corpora.

Three caveats matter more than the party trick itself:

1. It's a word-level example, not a document-level one. word2vec and GloVe give one fixed vector per word. Modern systems mostly use contextual embeddings from transformer models, where the same word gets a different vector depending on the sentence around it — "bank" near a river gets a different vector than "bank" near "loan."
2. The analogy works cleanly on curated examples, not on everything. Researchers have long noted these arithmetic relationships are approximate and don't hold with the same crispness across every word pair — it's a helpful mental model, not a guaranteed calculator.
3. It illustrates structure, not magic. The point for a PM isn't the parlor trick — it's the underlying claim: relationships between meanings can be represented as directions and distances in a coordinate space. That claim is what makes search, clustering, and recommendation systems work.

If you take one thing from the king-queen example, take this: embeddings don't store definitions the way a dictionary does. They store relative position — which is exactly what you need to answer "what's similar to this?" at scale.

A PM-Relevant Example: Grouping Duplicate Feature Requests

Say your feedback inbox has three tickets: "please add dark mode," "I want a night theme," and "can we get a dark UI option." A keyword search for "dark mode" only finds the first one. An embedding-based approach converts all three into vectors, measures their cosine similarity, and surfaces them as near-duplicates — because their meaning overlaps, not their wording.

This is a concrete version of a discovery problem every PM knows: feedback arrives in dozens of phrasings, and manually reading through hundreds of tickets to spot the same request in disguise doesn't scale. Clustering by embedding similarity is one of the more mundane, high-leverage applications of this whole field — it's synthesis work, the kind covered in our guide to running structured product discovery well.

A simplified workflow looks like this:

1. Embed every incoming request with the same embedding model, producing one vector per ticket.
2. Compare each new vector against existing ones using cosine similarity.
3. Threshold the result — anything above, say, 0.85 similarity gets flagged as a likely duplicate (the exact number is a tuning decision, not a fixed law).
4. Cluster the flagged group and route it to a human for the final call — a PM confirming "yes, these are the same ask" or splitting a false match.

That last step matters. Embedding similarity is a strong signal, not a verdict. Two requests can land close together because they share topic ("notifications") while asking for opposite things ("more notifications" vs. "fewer notifications"). Treating a high similarity score as an automatic merge, with no review step and no visible reasoning for why two tickets were grouped, is exactly the kind of silent, confusing failure mode worth designing around deliberately — see our piece on the UX of AI failure for the broader pattern.

Most mature systems don't pick one row — they combine keyword filters, embedding clustering, and a lightweight human review step.

Embeddings Are Not the Generative Model

An embedding model and a generative model do fundamentally different jobs, even though both are often built on similar transformer architecture. The embedding model's only output is a vector — a fixed-size fingerprint of meaning. The generative model's output is new text, produced one token at a time — and confusing the two is the most common misconception about AI search.

Both model types typically start from the same first step: breaking your text into tokens, the sub-word units models actually process — the same unit that drives your API bill, as we cover in tokens, not words: the AI billing unit. From there, the two jobs diverge sharply.

This distinction is why retrieval-augmented generation (RAG) is a two-step pipeline, not one model doing everything:

1. An embedding model turns your query and your knowledge base into vectors, and retrieval finds the closest matches.
2. A generative model reads those retrieved passages as context and writes an answer grounded in them.

If you want the fuller picture of how these pieces — tokens, context windows, weights, and generation — fit together into one mental model, our LLM fundamentals guide is the right primer to pair with this article. Retrieval also shows up as a first step inside many agentic workflows, where an agent looks up relevant context before deciding what action to take next.

Where This Shows Up in Real Products

Embeddings quietly power four product patterns a PM is likely to touch: semantic search, RAG, deduplication/clustering, and recommendations. All four reuse the same core mechanic — turn content into vectors, then find nearby ones — applied to a different surface of the product.

• Semantic search. Instead of matching keywords, the system embeds the query and every searchable item, then returns the closest vectors. This is why searching "cancel my plan" can surface a help article titled "How to end your subscription."
• Retrieval-augmented generation (RAG). Embeddings retrieve the most relevant internal documents or tickets; a generative model then writes an answer using them as grounding, reducing (though not eliminating) hallucination.
• Deduplication and clustering. As in the feature-request example above, similar items group automatically — useful for support tickets, survey responses, and roadmap items alike.
• Recommendations. "Users who engaged with this also engaged with…" often runs on embeddings of behavior or content, not just explicit ratings.

Under the hood, most production systems store these vectors in a vector database — purpose-built infrastructure like Pinecone or Meta AI's open-source FAISS library (Johnson, Douze, and Jégou), which can search millions of vectors for the closest matches in milliseconds using approximate nearest-neighbor (k-NN) algorithms. You don't need to know the math to manage a roadmap here, but knowing the term "vector database" exists will save you from a confusing engineering conversation.

Retrieval-augmented generation became a standard architecture pattern largely because embeddings solved the "how do we find the right 500 words out of a million-word knowledge base" problem cheaply and at scale.

A PM's Checklist for Evaluating an Embeddings Feature

Before greenlighting a search, RAG, or dedup feature, a PM needs answers to a handful of concrete questions — most of them tradeoffs, not right-or-wrong choices. Getting this wrong shows up later as slow search, ballooning storage costs, or embarrassingly bad matches.

Questions worth asking your engineering team early:

1. Which embedding model, and how many dimensions? More dimensions (say, 3,072 vs. 1,536) can capture finer nuance but cost more to store and search — a classic case for structured tradeoff analysis rather than a default "bigger is better" instinct.
2. How is quality being measured? Hugging Face's MTEB (Massive Text Embedding Benchmark), introduced by Muennighoff and colleagues in 2022, is the closest thing the field has to a standard scoreboard — it compares embedding models across dozens of retrieval, clustering, and classification tasks. A model that tops MTEB's general leaderboard still needs testing on your domain's vocabulary before you trust it.
3. What's the chunking strategy? Long documents get split into pieces before embedding; chunk size affects both retrieval precision and cost, and there's no universal right answer.
4. What happens on a near-miss? Decide, deliberately, what the product shows when the closest match is only 60% similar — silently returning it as if it were confident is a design choice, not a neutral default.
5. How will it be re-evaluated over time? Embedding quality can drift as your content and user language change; plan for periodic re-testing, not a one-time launch check.
6. What's the actual cost model? Embedding cost scales with tokens processed, and re-embedding your entire content library after a model upgrade is a real, sometimes-overlooked line item.

Where Prodinja Fits In

That's the honest scope of it — a prototype experience designed around the same coordinate-space idea this article explains, not a claim that it has already surfaced some specific result for anyone. If you're evaluating whether a "search that understands what you mean" feature belongs on your own roadmap, the checklist above is the same one worth running against your own product.

Key Takeaways

• Embeddings turn meaning into numbers. A vector is a fixed-length list of numbers positioned in a shared space, where distance between vectors approximates similarity of meaning.
• The king-queen analogy is a real, useful illustration — but it's a word-level party trick from word2vec/GloVe; modern contextual embeddings work at the sentence or document level and account for surrounding context.
• Grouping duplicate feature requests is a practical, everyday use case — embedding similarity finds paraphrased duplicates that keyword search misses, but still needs a human review step before merging.
• Embedding models and generative models do different jobs. One produces a vector for comparison; the other produces new text. RAG chains them together — retrieve with embeddings, then generate with an LLM.
• Vector databases (Pinecone, FAISS) and benchmarks (MTEB) are the infrastructure layer that makes searching millions of embeddings fast and lets teams compare model quality objectively.
• Choosing an embedding setup is a tradeoff exercise — dimensions, chunking strategy, cost, and re-evaluation cadence all need explicit decisions, not defaults.