Embeddings
Embeddings are lists of numbers that represent the meaning of a piece of text, or of an image or audio clip, so a computer can compare meanings with math. An embedding model turns “How do I reset my password?” into a list of hundreds or thousands of numbers. Texts that mean similar things get similar lists, even when they share no words: “I can’t log in” lands close to the password question, and “quarterly revenue report” lands far away.
Why it matters
Embeddings are what make search by meaning possible. Keyword search finds documents that contain the words you typed. Semantic search embeds your question and every document, then returns the documents whose embeddings are closest to the question’s. That’s how an AI assistant finds the right policy page when you ask in your own words.
They’re also the basis of retrieval-augmented generation (RAG). A company’s documents are split into chunks, each chunk is embedded and stored in a vector database, and when someone asks a question, the closest chunks are pulled into the model’s context. Embeddings are used for recommendations, grouping similar support tickets and finding duplicates too.
Where it goes wrong
- Similar isn’t always relevant. A question about cancelling a contract can pull up a passage about cancelling a meeting. Many systems combine embeddings with keyword search and a reranking step to catch this.
- Exact terms get blurred. Product codes, names and numbers often embed poorly, so a search for one specific part number can return near misses.
- Chunking matters. If a chunk cuts a table or a clause in half, its embedding only captures half the meaning.
- Models don’t mix. Embeddings from different models can’t be compared, so changing your embedding model means embedding everything again.