At the end of module 3 we left DocuBot in a curious position: it is a robust system, with memory, financially monitored... and blind. It answers beautifully when we hand it the right documentation, but so far that hand has been ours: in every example, we were the ones pasting the relevant fragments into the <documentation> block. The open question was: who finds, among hundreds of pages of Nubelia's wiki, the two fragments that answer each question? This lesson builds the first piece of the answer: embeddings — which in 01-02 we introduced intuitively as "coordinates of meaning" — go from intuition to code today. You will learn to obtain them via API, to measure how similar two texts are with cosine similarity, and to build a working (if brute-force) semantic search over Nubelia's documentation. And as a bonus, we will fix the limitation we announced in 03-04: the exact-match ResponseCache that couldn't recognize equivalent questions phrased differently.

1. From "coordinates of meaning" to real vectors
2. Getting embeddings via the API
3. Cosine similarity: measuring how close two meanings are
4. Brute-force semantic search over the Nubelia docs
5. Keywords vs. meaning: why semantic search wins
6. Choosing an embeddings model
7. Bonus application: the semantic response cache

In 01-02 we saw the intuition with the dog/cat/invoice example: an embedding places each text on a map where similar meanings end up close together. "Dog" and "cat" are neighbors; "invoice" lives in another neighborhood. Today we make that metaphor concrete:

• An embedding is a list of decimal numbers (a vector) that represents the meaning of a text. Typical lengths: between 256 and 3072 numbers (its dimensions).
• It is produced by an embeddings model, which is a different model from the LLM that generates text. It doesn't converse: it only turns text into coordinates.
• The key property: two texts with similar meaning produce nearby vectors, even if they don't share a single word. "I can't log in" and "reset password" end up close on the map; "export a report" is far from both.

For DocuBot this changes everything: if we convert every fragment of Nubelia's documentation into a vector, and the user's question too, finding the relevant documentation reduces to a geometry problem — finding the vectors closest to the question's.

Just as in 03-01 with the LLM, we use a managed API. All embeddings providers follow the same pattern: you send a list of texts, you receive a list of vectors. Here we show it with the voyageai SDK as an example, but the code is deliberately generic — switching providers means changing three lines:

A breakdown for anyone seeing this for the first time:

• embed() takes a list, not a single text: embeddings APIs are designed to work in batches, and embedding 50 texts in one call is far cheaper in latency than 50 single-text calls. We will exploit this during ingestion (04-03).
• input_type: several models distinguish between embedding a document (which will be searched) and a query (which searches). If your provider doesn't support it, ignore the parameter; if it does, use it — it improves retrieval.
• The result is a list of vectors; each vector is a list[float] with as many dimensions as the model produces (e.g. 1024).

One important detail: no single number in the vector means anything on its own. The meaning lies in the vector's overall position relative to the others. That is why the only useful operation is comparing them.

We need a number that says "how similar" two vectors are. The standard is cosine similarity, and the geometric intuition is simple: picture each vector as an arrow starting at the origin of the meaning map. Cosine similarity measures the angle between the two arrows:

• Arrows pointing in the same direction (angle 0°) → similarity 1.0: nearly identical meanings.
• Perpendicular arrows (90°) → similarity 0.0: unrelated meanings.
• Opposite arrows → similarity -1.0 (rare in practice with real texts).

The interesting part is that it measures direction, not length: a long paragraph and a short sentence about the same topic point the same way. The implementation fits in ten lines and needs no math beyond multiplying and adding:

• The dot product (sum(x*y)) grows when the two vectors have large values in the same positions — that is, when they "point the same way".
• Dividing by the two norms (lengths) removes the size effect: only the angle remains.
• In production you would use numpy for speed, but this pure version makes it clear there is no magic.

The exact values depend on the model (each model has its own typical "scale"), but the relative order is reliable: related things score higher than unrelated ones. Hold on to this idea — it will resurface when we calibrate thresholds.

We now have both pieces. The simplest possible semantic search — brute force — consists of: embedding all the fragments once, embedding the question, computing the similarity against all the fragments and keeping the best k (top-k):

Typical output:

Notice what just happened: the question "I can't log in" shares no keywords at all with "reset password" or with "account gets locked after several failed attempts", and yet the search found exactly the two fragments a human support agent would have picked. This is what no keyword search does well.

Two observations before moving on:

• Step 1 (embedding the corpus) is done once and reused; only the question is embedded on each search. Even so, keeping the vectors in a Python list has serious scale and persistence problems — that is exactly the topic of the next lesson (04-02).
• Here the "fragments" were handed to us ready-made. Deciding how to split hundreds of real pages into good fragments is a craft with a name of its own, chunking, and it gets its own lesson (04-03).

The classic alternative is keyword search (the grep kind, the basic wiki search kind): finding documents that contain the same words as the query. Let's compare:

The table reveals something important: semantics is not strictly superior at everything. For exact error codes or function names in the Nubelia API, literal search is hard to beat. That is why many mature systems use hybrid search (combining both) — knowing it exists is enough for now; for DocuBot, pure semantic search covers the vast majority of support questions, which arrive written in the user's words, not the documentation's.

A frequent mistake is assuming the embeddings model "comes with" the LLM. They are independent decisions: DocuBot can generate answers with the mid-size model chosen in 01-03 and embed with a model from another provider. The selection criteria:

And a golden rule worth tattooing: vectors from different models are not comparable with each other. If you switch embeddings models tomorrow, every vector in the corpus becomes obsolete and everything must be re-embedded. That is why EMBEDDINGS_MODEL is a constant with an env var, and why the ingestion in 04-03 will be re-runnable.

In 03-04 we built ResponseCache: an exact cache keyed by the SHA-256 hash of the normalized question, with a TTL. And we announced its limitation: "how do I reset my password?" and "I forgot my password, what do I do?" produce different hashes → two LLM calls for the same answer. With what we learned today, the solution is direct: also cache the vector of each question and, before calling the LLM, check whether any cached question is similar enough:

Design points that deserve attention:

• The threshold is the critical decision. At 0.92 we require nearly equivalent questions. If you lower it to 0.80, you save more calls but risk serving the answer to "how do I invite a member?" to someone who asked "how do I remove a member?" — questions that are topically close but have opposite answers. One false positive in a cache is worse than a hundred misses: the user receives a wrong answer with total confidence. Calibrate it with the evaluation dataset from 02-04, starting conservative.
• There is still a cost: every cache lookup embeds the question (one cheap call to the embeddings API). The net savings come from avoiding the expensive LLM call. With the relative prices from pricing.py (03-04), the math works out heavily in our favor.
• The search inside the cache is brute force — it walks all the entries. For a cache of hundreds of questions this is perfectly fine; it is the same pattern as this lesson's search(). (And yes: if the cache grew to millions of entries, it would need the same thing the corpus does — a vector database. It all connects.)
• Invalidation still applies: when Nubelia publishes new documentation, the cache is flushed, exactly as we decided in 03-04.

• Comparing vectors from different models. Produces meaningless numbers, with no visible error. Pin the model with EMBEDDINGS_MODEL and treat it as part of your data schema.
• Interpreting similarity as a universal probability. A 0.75 does not mean "75% relevant": the scale depends on the model. Always compare within the same model, and calibrate thresholds empirically with your evaluation dataset.
• Forgetting input_type (in models that support it): embedding questions as if they were documents silently degrades retrieval.
• Embedding text by text in a loop. The APIs accept batches; embedding one at a time multiplies latency and sometimes cost. Save the loop for the user's question (which inevitably arrives one at a time).
• A semantic cache threshold that is too generous. Start at 0.90+, measure false positives with cases of the "similar questions with different answers" kind (the soft adversarial cases from 02-04 are good candidates), and lower it only with evidence.
• Tip: store each vector alongside the text that produced it. A vector without its text is unrecoverable — there is no "un-embed" operation.

1. Top-k with a minimum threshold. Modify search() so it accepts a min_threshold parameter and discards fragments whose similarity falls below it, even if that leaves fewer than k results. Try it with the question "Can I pay with cryptocurrency?" (which has no answer in the mini-corpus) and observe what it returns.
2. Keywords vs. semantics. Implement a function keyword_search(question, k) that scores each fragment by the number of words it shares with the question (ignoring case and words shorter than 4 letters). Compare it with search() for the questions "I can't log in" and "API token": which one wins in each case, and why?
3. Calibrating the semantic cache. With SemanticCache, save the answer to "How do I reset my password?" and try to retrieve it with: (a) "I forgot my password, what do I do?", (b) "How do I change my password?", (c) "How do I invite a user?". Try thresholds 0.80, 0.90 and 0.95 and record in a table which combinations hit and which produce false positives.

1. Filtering before slicing is enough:

With "Can I pay with cryptocurrency?" and a reasonable threshold (e.g. 0.5), the list comes back empty or nearly so: there is no relevant documentation, and returning an empty list is more honest than returning noise. Keep this idea — in 04-04 it will be the piece that triggers the anti-hallucination phrase from SYSTEM_DOCUBOT ("I can't find that information in the available documentation.") when retrieval finds nothing worthwhile.

2. A minimal version:

Expected result: for "I can't log in", keyword_search scores essentially everything at 0 (no word of four or more letters survives the filter) while semantic search nails it. For "API token", keyword search wins or ties: "token" is an exact term that appears literally in the right fragment (note that "API" itself gets dropped by the 4-letter rule — a real tokenizer would keep it). Conclusion: semantics dominates for natural user language; literal matching remains strong for exact terminology.

3. Typical results (yours will vary by model):

At 0.95 the cache barely saves anything; at 0.80 it hits more often but the safety margin against "similar questions with different answers" pairs shrinks. 0.90 is usually the reasonable starting compromise — and the final figure must come from measuring against the evaluation dataset, not from this exercise.

Today embeddings stopped being a metaphor from 01-02 and became three functions that fit on one screen: embed() turns text into coordinates of meaning, cosine_similarity() measures the angle between two of those coordinates, and search() finds the top-k fragments of Nubelia's documentation closest to any question — including "I can't log in", which shares not one word with the answer it needs. We saw why this beats keyword search (and when it doesn't), which criteria matter when choosing an embeddings model (independent of the LLM, with vectors incompatible across models), and we collected the bonus promised in 03-04: SemanticCache, which recognizes equivalent questions by meaning with a carefully calibrated threshold. But our search has two embarrassments we swept under the rug: it compares the question against all the vectors on every query, and those vectors live in a Python list that vanishes when the process restarts — with hundreds of pages of real documentation, neither is acceptable. Solving both is the job of vector databases, and that is exactly what we build in the next lesson (04-02).