Vector search and similarity

Vector search finds text by nearness in meaning rather than by matching the exact words, which makes it excellent for prose and quietly wrong for code, where the piece of code that calls a function and the function itself often share almost no vocabulary at all.

Searching by words, and why it is not enough

The oldest way to search text is to match characters. You type a word, the tool finds every place those exact letters appear. This is how the classic command line search grep works, and it is how a plain find box in an editor works. It is fast and it is honest: it returns precisely what you asked for, no more and no less. For many jobs that is exactly right.

But word matching has a blind spot that anyone who has used a search box has felt. It has no idea what words mean. Search for car and you will not find a page that only ever says automobile. Search for how do I cancel my subscription and a word matcher will hunt for those literal words, missing the help article titled ending your plan that answers the question perfectly without using a single one of them. People rarely phrase a question the way the answer is written. Word matching demands they do, and punishes them when they do not.

What we usually want is not documents that share words with the question, but documents that share meaning with it. That is a harder thing to compute, because meaning is not sitting in the characters. To search by meaning, you first need a way to turn a piece of text into something that captures what it is about, in a form a computer can compare. That form is a vector.

Turning meaning into a position

A vector is just a list of numbers. The trick that makes vector search work is a model, called an embedding model, that reads a piece of text and produces a list of numbers standing for its meaning. Text that means similar things gets similar lists of numbers. Text that means different things gets different lists. The full story of how that is possible is the subject of embeddings and vector space, but you can carry the essential picture with a simple image.

Picture a map. On a real map, two towns that are close in the world are close on the paper, and their closeness is something you can measure with a ruler. Now imagine a map not of places but of meanings, where every possible sentence has a location. On this map, the sentence the cat sat on the mat and the sentence a feline rested on the rug land near each other, even though they share hardly any words, because they mean nearly the same thing. A sentence about interest rates lands far away in another region entirely. A vector is the coordinates of a piece of text on this map of meanings. It usually has hundreds of numbers rather than the two you would need for a paper map, but the idea is the same: a position, and positions can be near or far.

Once text has a position, searching becomes measuring distance. This is worth saying slowly because it is the whole mechanism.

1. Ahead of time, run every document through the embedding model and store its vector. This is your searchable collection of positions.
2. When a question arrives, run the question through the same embedding model to get its vector, its position on the map.
3. Find the stored vectors nearest to the question’s vector. Those nearest neighbors are the documents closest in meaning to the question.
4. Return them, usually the closest handful, in order of nearness.

That is vector search in full. Embed the query, find the nearest neighbors, return them. The cancel my subscription question and the ending your plan article land near each other on the map of meanings, so the article comes back even though they share no words. This is exactly what a word matcher could not do.

Finding the nearest without checking everything

There is a practical problem hiding in step three. If you have a million documents, finding the true nearest neighbor by comparing the question to every single one is a lot of work, and it has to happen fast enough to feel instant. Doing it exactly, every time, does not scale well.

The common answer is to give up a little accuracy for a large gain in speed. Instead of guaranteeing the exact nearest neighbors, these systems find the almost-nearest neighbors, nearly always the right ones, in a tiny fraction of the time. This is called approximate nearest neighbor search. You do not need the internals, only the intuition, and an everyday analogy carries it.

Suppose you are dropped into a large city and asked to find the nearest coffee shop. You could walk to every coffee shop in the city and measure, which is exact but absurd. Instead you note which neighborhood you are in, go to that neighborhood, and check the few shops there. You might, once in a while, miss a shop just over the border of the next neighborhood that was technically a bit closer. Almost always you find the nearest one, and you find it in minutes instead of days. Approximate nearest neighbor search does the same thing with meanings: it organizes the map into regions so that, at question time, it only has to look carefully in the promising regions rather than the entire map. The result is search that stays fast even over enormous collections, at the cost of an occasional near miss that rarely matters.

Where vector search shines

For ordinary human language, vector search is a genuine step up, and it is fair to be enthusiastic about it in its home territory. Prose is written by people for people, and people say the same thing in endless different ways. The strength of vector search is that it forgives that variety. It connects a question to an answer that means the same, regardless of the exact words each one chose.

• Support and help systems, where users describe a problem in their own words and the fix is written in someone else’s.
• Searching notes, articles, and documentation by idea, so that a search for burnout also surfaces a piece titled staying rested at work.
• Grouping similar items, like finding news stories about the same event even when no two headlines are worded alike.
• Feeding a language model relevant passages before it answers, which is the retrieval step in retrieval-augmented generation.

In all of these, the thing that connects the query to the right result is shared meaning carried by shared or synonymous words. The two ends of a good match tend to talk about the same subject in overlapping language. That overlap is the fuel vector search runs on. Where the fuel is present, it works beautifully.

The sharp failure: when meaning does not live in the words

Now to the honest limit, and it is a big one for a particular kind of data. Vector search rests on an assumption so quiet it is easy to miss: that things which belong together use similar words. For prose that assumption usually holds. For code it often does not, and where it breaks, vector search fails in a way that is hard to notice because it still returns confident-looking results.

Consider the most basic relationship in software: one piece of code uses another. A function named parseInvoice might do its real work by calling a helper named normalize. The connection between them is central. If you change how normalize behaves, parseInvoice can break. Yet look at the words. One is about invoices, the other is about normalizing. They share no vocabulary. Their names come from different corners of the problem. To an embedding model, which places text by the meaning of its words, these two land in different regions of the map, far apart, even though in the running program they are as tightly bound as two links in a chain.

Turn it around and it is just as bad. Two functions can sit near each other on the map because they use similar words, while having nothing to do with each other in the actual program. A function called sendEmail and another called sendSms will look similar to an embedding model, because their names and bodies talk about sending messages. But changing one need not touch the other at all. They resemble each other in language and are unrelated in wiring.

In code, the pieces that depend on each other often share no words, and the pieces that share words often do not depend on each other. Similarity and dependency come apart.

This is the crux. In prose, resemblance in words is a decent proxy for the relationship you care about. In code, the relationship you care about, which function calls which, which file imports which, which service talks to which, is a fact about wiring, not a fact about vocabulary. Vector search measures resemblance in words. So when you ask the question that matters most about code, if I change this, what else is affected, similarity search gives you things that read like your query and misses the things that actually depend on it. It is not a little bit off. It is answering a different question than the one you asked, and doing so fluently enough that you might not catch the substitution.

A worked contrast

Imagine you are about to change normalize and you want to know everything that could break. What you actually want is the list of callers: every place in the codebase that uses normalize, directly or through a chain. That is a precise, checkable set, and it is defined by real links, not by resemblance.

Ask a vector search for code similar to normalize and you will get other small utility functions that look like it: helpers with short names and tidy bodies, perhaps a sanitize and a trim somewhere far away that have never once called normalize. Meanwhile parseInvoice, the function that will genuinely break when you change normalize, may not appear at all, because it does not resemble normalize in words. The search returned lookalikes and hid the real dependents. For prose this would be a minor annoyance. For a change you are about to ship, it is the difference between seeing the blast radius and being blindsided by it.

Why this points toward a graph

The lesson is not that vector search is bad. It is that vector search answers a specific question, find me things that mean something similar, and that question is the wrong one when what you need are the real connections in a system. When the relationships are explicit, function A calls function B, file X imports module Y, service P sends a request to service Q, you should not be inferring them from word overlap. They are already there, stated exactly in the code. You should be following them.

Following explicit links is a different shape of retrieval altogether. Instead of a map where you measure distance, you want a network of nodes joined by edges, where each edge is a real relationship you can walk along. Start at normalize, walk the called-by edges outward, and you arrive exactly at parseInvoice and every other true caller, with no lookalikes and no misses. That structure, the codebase drawn as things joined by their real relationships, is the code knowledge graph, and retrieval that walks it rather than measuring similarity is the subject of GraphRAG for code. Vector search shows why that structure is needed: for code, nearness in meaning is simply not the same as being connected.