Embeddings and vectors

This is the fundamentals question most freshers fumble, and it is entirely learnable in fifteen minutes.

The problem embeddings solve

A computer can tell you whether two strings are equal. It cannot tell you that "how much is the hostel fee" and "what do I pay for accommodation" are the same question. Keyword search fails here: the two share almost no words.

An embedding is a list of numbers — a vector — that represents a piece of text's meaning in a way a computer can compare. Typically a few hundred to a few thousand numbers:

"how much is the hostel fee"   → [ 0.021, -0.144,  0.087, ... ]   (768 numbers)
"what do I pay for accommodation" → [ 0.019, -0.139,  0.091, ... ]   (768 numbers)
"the mitochondrion is an organelle" → [-0.201,  0.310, -0.055, ... ]

The first two vectors are close together. The third is far away. Nobody wrote a rule saying "hostel" relates to "accommodation" — a model was trained such that text with similar meaning lands in similar places.

That is the whole idea: an embedding turns "do these mean the same thing?" into "are these two lists of numbers pointing in the same direction?", which is arithmetic, and arithmetic is searchable.

Why direction, not distance

The standard comparison is cosine similarity — the cosine of the angle between two vectors. It runs from 1 (same direction, same meaning) through 0 (unrelated) to -1 (opposite).

Why the angle rather than straight-line distance? Because vector magnitude tends to track things you do not care about, like how long the passage was. A one-line answer and a three-paragraph answer about the same topic point the same way but have different lengths. Cosine similarity ignores that; Euclidean distance does not.

cos(a, b) = (a · b) / (|a| × |b|)

The division by both magnitudes is what discards length. Worth knowing: if vectors are normalised to length 1 — as many embedding APIs return them — then cosine similarity and dot product are the same computation, which is why you will see both in real code.

What a vector database is for

If you have 50,000 passages and a query, you could compute cosine similarity against all 50,000 and sort. For 50,000 that is genuinely fine. For 50 million it is not.

A vector database (FAISS, Pinecone, Chroma, pgvector, and others) exists to answer "which vectors are nearest to this one?" quickly, using an approximate nearest neighbour index. Note the word approximate: it trades a small amount of accuracy for a large amount of speed. It may occasionally miss the true closest match.

Say that word "approximate" out loud in an interview. It signals that you know a vector database is an indexing optimisation, not a magic meaning-machine.

Chunking: the choice you will be asked about

You cannot embed a whole 40-page PDF as one vector — meaning would be averaged into mush, and you could not tell the model which part was relevant. So documents are split into chunks and each chunk is embedded.

The size is a genuine trade-off, and this is the most common "why did you choose that?" question in the whole area:

• Chunks too small (a sentence) — each vector is precise, but a retrieved chunk lacks the context needed to answer. You get the line "the fee is ₹45,000" with no indication of what fee.
• Chunks too large (a whole page) — plenty of context, but the vector represents several topics at once, so it matches queries about all of them weakly and none of them strongly.

A few hundred tokens with a small overlap between neighbours is the common starting point. The overlap exists so a sentence sitting on a boundary is not orphaned. The honest interview answer is that it depends on your documents and you would tune it by testing retrieval quality — which is also true, and much better than a made-up number.

The two follow-ups that catch people out

"Can you go from the embedding back to the text?" No. Embedding is lossy and one-directional. The vector is a representation, not an encoding. This is why every real system stores the original text alongside the vector — retrieval finds the vector, then you look up the text it came from.

"What happens if you change the embedding model?" Every stored vector becomes useless. Vectors from two different models are not comparable — the numbers mean different things. You must re-embed your entire corpus. Students who only followed a tutorial almost never know this, and it is a real operational consequence, which is exactly why it gets asked.

What to be able to say

An embedding is a vector that represents meaning, produced by a model trained so that similar text lands close together. I compare them with cosine similarity, which measures the angle so passage length doesn't distort the result. A vector database indexes those vectors so nearest-neighbour search is fast — approximately, in exchange for speed. I chunk documents before embedding, because one vector cannot represent forty pages, and chunk size is a trade-off between precision and having enough context to answer.

Next lesson: putting this together into a pipeline, and the diagram you will be asked to draw.