AI 101 — What is an embedding?

An embedding is a list of numbers that represents the meaning of something — a word, a sentence, a photo, a song — so a computer can compare meanings using arithmetic. The rule is simple and surprisingly powerful: things that mean similar things get similar lists of numbers. That's the whole idea, and it's the foundation of semantic search, recommendation feeds, and most of what people call "AI memory."

Why it matters right now

Embeddings are the invisible layer under nearly every AI feature that touches your own data. When an agent "remembers" what you discussed last week, when a support bot finds the right paragraph in a manual, when a photo app groups pictures of the same dog — an embedding did the matching. The models that produce them are also having a moment: MLCommons just added a vector database test to MLPerf Storage, and vector search systems are now being stress-tested at 10-billion-document scale, so the infrastructure built around embeddings is finally getting measured the way training hardware has been for years.

That matters because these numbers are the bridge between language and math. Text is messy; numbers can be compared, sorted, clustered, and searched in milliseconds. Embeddings are how meaning becomes something a machine can do arithmetic on.

The mental model

Take a sentence and run it through an embedding model. Out comes a fixed-length list of numbers — a vector — typically somewhere between a few hundred and a few thousand entries long. Google's open EmbeddingGemma, for example, turns any passage into 768 numbers. The individual numbers don't mean anything you could label; there's no "number 42 = dogs." Meaning lives in the pattern across the whole list, and in the distances between different lists.

Picture every passage as a dot in a vast space with hundreds of directions to move in. During training, the model gets pushed to place related passages close together and unrelated ones far apart. Once that's done, "how similar are these two things?" becomes "how close are these two dots?" — measured with cosine similarity, a standard way of scoring how much two lists of numbers point in the same direction. Search becomes: turn the question into a dot, find the nearest stored dots, return those documents.

An everyday analogy

Imagine a huge pantry where nothing is labeled with words, only with coordinates. During setup, someone tasted every jar and shelved them by flavor: sweet things together, smoky things in another cluster, bitter things in a corner. Nothing says "honey" on the shelf — but the jars of honey all ended up side by side, near the maple syrup and farther from the vinegar.

Now you walk in and ask for "something sweet to put on porridge." Your request gets turned into coordinates too, and the nearest jars are clover honey, maple syrup, and date syrup. No keyword matched the word "porridge," because no label mentioned porridge. The system found what you meant. That is what an embedding does — and why it beats keyword search when you don't know the exact words in the document you need.

Common misconceptions

"Embeddings are just fancy keyword counts." They're the opposite. Two sentences can share every important word and have different embeddings if the meanings differ, and can share no words at all and sit next to each other if they mean the same thing. That's why searching by meaning works when searching by vocabulary doesn't.

"The numbers are a secret code you can decode." There's no key. Each number is a coordinate learned during training, not a symbol. Reading one tells you nothing; only the whole vector, compared against another, carries information.

"Bigger is always better." Longer vectors can carry more nuance, but they cost more to store, index, and search. Many models now let you deliberately shorten them — EmbeddingGemma, for instance, keeps most of its quality at a third of its full length, which is the same trick as a lower-resolution photo that still shows what you're looking for.

"Embeddings understand truth." They capture relatedness, not correctness. A well-written lie and a well-written truth on the same topic will sit close together. Embeddings find relevant material; they don't vouch for it.

Where to learn more

OpenAI's embeddings guide is a clear starting point for how the vectors are produced and compared, and Hugging Face's EmbeddingGemma post shows a small open model you can actually run yourself. To see where these vectors go next, read What is a vector database? — the warehouse that indexes them at scale — and What is RAG?, the pipeline that retrieves them and hands them to a model inside its context window. For the underlying units those vectors are built from, start with What is a token in AI?.

Related reading: What is a vector database? · What is RAG? · What is a token in AI?

If you've built something with embeddings, what surprised you most — how well it worked, or where it quietly failed? Tell us in the comments.

Sources: OpenAI — Vector embeddings guide · Hugging Face — EmbeddingGemma · Google — EmbeddingGemma technical report (arXiv) · MLCommons — MLPerf Storage v3.0 · Qdrant — 10-billion-vector benchmark