So far we've seen what generative AI can do and how to evaluate the available models. This lesson completes the module with the part no professional developer can skip: what can go wrong. LLMs invent information with total confidence, have a limited working memory, ignore everything after their training, don't answer the same way twice, carry biases, and cost money on every call. None of this makes them useless — but each limitation must shape your application's design from day one. We'll use DocuBot and Nubelia as a mental test bench for each risk: understanding them now is what will let us, in the following modules, build the mitigations knowing exactly why.

1. Hallucinations: when the model invents with confidence
2. The context window and its implications
3. The knowledge cutoff
4. Non-determinism: same prompt, different answers
5. Biases in the data and in the responses
6. Costs and consumption as a design constraint
7. Privacy: what happens to the data you send

A hallucination is a false or invented answer that the model presents with the same fluency and apparent confidence as a correct one. It's the most important limitation of LLMs and the first one you must internalize.

Recall the mechanism from lesson 01-02: an LLM generates the statistically most plausible text to follow what it has received. Nowhere in that process is there any fact-checking. Three factors combine:

1. The model optimizes for plausibility, not truth. An invented but reasonable figure is perfectly plausible text.
2. It has no reliable internal "I don't know" mechanism. Faced with a question whose content isn't in its data, the most plausible continuation is still an answer, not silence.
3. Training with human feedback (RLHF) rewards responses that sound helpful and confident, which can amplify the tendency to answer with confidence rather than admit ignorance.

Picture the "naive" DocuBot from lesson 01-01: a generic LLM hooked up to Nubelia's chat, with no access to the documentation. Examples of the kind of dialogue we'd see (fictional, of course, like everything about Nubelia):

A fluent, specific answer, complete with menu paths... and entirely made up. The model knows nothing about Nubelia; it has generated what tends to be true in similar SaaS products. Maybe Nubelia exports to PDF, maybe it doesn't — the model doesn't know, but it answers as if it did. Worse still:

An impeccable-looking endpoint... for a feature that may not exist. A trusting developer would lose an hour before discovering the deception. This is the dangerous pattern: the more plausible the domain, the more convincing the hallucinations.

It's important to understand that hallucinations aren't limited to private data: models also invent things about public knowledge (quotations, library functions, bibliographic references), especially in the fine details and in topics underrepresented in their data.

There is (today) no solution that eliminates hallucinations entirely, but there is a set of mitigations that reduce them drastically. We present them now and will build them throughout the course:

Notice that the first mitigation is exactly the reason RAG exists: the difference between the naive DocuBot and the course's final DocuBot will be that the latter answers with Nubelia's real documentation in front of it — and citing it.

The context window is the maximum number of tokens the model can handle in one call: instructions + history + documents + the generated response itself — it all adds up. It is, literally, everything the model "sees" when responding; outside it, nothing exists.

Design implications:

• You can't pass in everything. Nubelia's full documentation exceeds any reasonable window (and even if it fit, you'd pay for all those tokens on every call). You have to select what goes in: that's the problem RAG's retrieval solves (module 4).
• Conversations don't fit forever. In a chat, every turn accumulates. Sooner or later you have to truncate, summarize or discard history — the strategies are covered in 03-03.
• The model doesn't "remember" between calls. There is no persistent memory: if you want DocuBot to remember the conversation, your application must resend the history on every call. A chatbot's "memory" is your engineering, not a capability of the model.
• A big window ≠ perfect attention. With very long contexts, models can make poorer use of information located in the middle of the text (a phenomenon known as lost in the middle). More context also means more cost and latency. The quality of what you put in the context matters more than the quantity.

As we saw in 01-02, the model's knowledge comes from its pre-training and is frozen at a date: the knowledge cutoff. Everything after it — new library versions, events, changes to public APIs — simply isn't there.

Practical consequences:

• Questions about "the latest" in any technology can get outdated answers with no warning signal whatsoever: the model confidently describes the old version.
• Combined with hallucinations, it's treacherous: asked about something after its cutoff, the model can blend what it knew with invented extrapolations.
• For DocuBot there's an essential nuance: even if we retrained the model on Nubelia's documentation, it would go stale with every release. Documentation changes every week; models are trained every many months. That's why the right strategy is not "putting the knowledge into the model" but passing the current information in the context on every call — once again, the module 4 RAG pattern.

Design rule: for any data that changes over time, treat the model's internal knowledge as potentially expired and supply the current version in the context.

An LLM is not a pure function: the same input can produce different outputs on different calls. We saw it in 01-02: sampling deliberately introduces randomness, and not even temperature = 0 gives an absolute guarantee of reproducibility (due, among other things, to details of parallel GPU arithmetic and model version changes at the provider).

What this means for you as a developer:

• Testing: you can't assert response == "expected text". Tests of LLM applications verify properties (does it contain the right citation? is it valid JSON? does it mention feature X?) or use evaluators — we'll see this in 06-03.
• Debugging: a failure may not reproduce on the first try. Logging the inputs and outputs of every call (observability, 06-04) stops being optional.
• User experience: two Nubelia employees asking DocuBot the same question will get differently worded answers. Acceptable in general — but if the substance varies (it tells one of them yes and the other no), you have a reliability problem that evaluation must catch.
• Silent model changes: providers update and deprecate models. The version you evaluated today may behave differently after an update: pin versions when possible and re-evaluate when you change them.

LLMs learn from text written by people, with all its biases: cultural, gender, geographic, linguistic. The model absorbs them and can reproduce or amplify them.

Manifestations relevant to real applications:

• Social biases: stereotyped associations when generating descriptions, examples or assessments involving people or groups.
• Language bias: performance is strongest in English (dominant in the training data); in less-represented languages the same model can reason somewhat worse or produce awkward, English-calqued phrasing. Nubelia's employees mostly write in English, but the questions that arrive in other languages are exactly where quality can dip.
• Availability bias: whatever is most frequent in the data becomes the "default" answer — popular technical solutions recommended even when they're not right for your case.
• Sycophancy: a tendency to agree with the user and adapt to their expectations, instead of contradicting them when warranted.

Even in a "purely technical" assistant like DocuBot this needs watching: if the support team asks it to draft replies to customers, biases in tone and assumptions about the user end up in real communications. High-level mitigation: explicit instructions on tone and assumptions (module 2), human review of any text leaving the company, and evaluation with diverse cases (module 6).

Every LLM call costs money (input + output tokens, as we saw in 01-03) and consumes time and energy. We treat it here as a design risk — detailed operational management (caching, optimization, spend monitoring) arrives in 03-04.

The risk has a concrete shape: the cost of LLM applications scales with usage in ways traditional software doesn't. A classic REST endpoint costs practically the same at 10 or 10,000 requests a day; with an LLM, 1,000× more requests is (roughly) 1,000× more cost. Typical design mistakes that blow it up:

• Bloated context: resending unnecessary documentation or history on every call. Remember: you pay for all input tokens on every call.
• Avoidable calls: using the LLM for what a database query or a simple rule would solve.
• Uncontrolled loops and retries: a badly handled failure that retries in a loop is money pouring out of an open tap (we'll handle this in 03-02).
• Oversizing the model: using the large tier for small-tier tasks (lesson 01-03).

A napkin estimate for DocuBot, to build the reflex:

The exact figure matters little (prices change); the habit does: do this math at the design stage of any LLM feature, not when the bill arrives.

The last limitation, and often the first thing your company will ask about: when you use a model via API, the content of every call travels to the provider. Prompts, documents included in the context, users' questions — everything.

Key points at a high level (the regulatory detail — GDPR, data-processing agreements, data residency — is covered in lesson 06-02):

• Read the provider's data terms: does it use your calls to train models? (the enterprise API plans of serious providers, as a general rule, do not — unlike some free consumer products); how long does it retain the data?; where is it processed?
• Minimize what you send: don't include personal or confidential data in the context that the task doesn't need. If a support ticket contains a customer's data and the LLM only needs to classify it, anonymize or strip that data before the call.
• Beware of "shadow AI": employees pasting confidential information into consumer AI chats, outside all corporate control. Part of the value of building an official DocuBot is precisely offering an approved, controlled channel.
• For Nubelia: the internal documentation is confidential but contains no customer personal data, so an API with good contractual guarantees is acceptable (as we decided in 01-03). If DocuBot ever processed tickets containing customer data, the privacy analysis would have to be redone — and that's a compliance decision, not just a technical one.

Deliberately left out of this lesson is the other big security front — users manipulating the model through prompt injection and related attacks — which has its own lesson (06-01).

• Trusting an answer because it sounds confident. The fluency of the text doesn't correlate with its truthfulness; neither does the confidence of the tone. Verify every actionable piece of data (endpoints, commands, figures) against the source before using it.
• Trying to fix hallucinations with the prompt alone. Instructions like "don't make things up" help, but they're not enough: without the right information in the context, the model has no alternative to its frozen knowledge. The real mitigation is architectural (RAG), not cosmetic.
• Designing as if the model remembered. "I told it three messages ago" doesn't exist for the model if your application didn't resend those messages. Managing the history is your code's responsibility.
• Discovering the costs on the invoice. Estimate cost per call × volume at the design stage, and set spending limits and alerts from day one.
• Sending sensitive data "because it's more convenient". Ask yourself before every call: does the model need this data for the task? If not, out it goes. Minimization is privacy's cheapest rule.
• Assuming these limitations will vanish next month. Models improve, but hallucinations, finite context, knowledge cutoff and per-token cost are structural characteristics of today's technology. Design with them, not against them.
• Tip: turn this lesson into a design checklist. For every new LLM feature: what happens if it hallucinates? does the context fit? is the data current? can I tolerate variability? are there biases with impact? what does it cost at my volume? what data leaves, and should it?

For each DocuBot incident, identify which limitation from the lesson explains it and one reasonable mitigation:

1. DocuBot explains in detail how to enable Nubelia's "calendar integration". That integration doesn't exist; there's a similar one with a different name.
2. After 40 minutes of conversation, DocuBot starts contradicting things it said itself at the beginning of the chat.
3. An employee asks about the deployment policy approved last month and DocuBot describes the policy from two years ago that appeared on a public website.
4. The QA team runs the same test twice and the response cites the correct source the first time, but not the second.

Nubelia's product team proposes that DocuBot, besides answering questions, automatically summarize every support ticket when it's created (about 3,000 tickets/month, average input 2,000 tokens, output 200). With the lesson's illustrative prices (€3/M input, €15/M output): calculate the added monthly cost, and mention one design decision that would reduce it without dropping the feature.

Write the short text (2–4 sentences) that DocuBot should display permanently in its interface to set the right expectations for Nubelia's employees, covering at least: the possibility of error, the need to verify critical information, and what not to paste into the chat.

Solution 1:

1. Hallucination (aggravated by the plausibility of the domain: a "calendar integration" is typical of a project-management SaaS). Mitigation: answer only from fragments of the real documentation retrieved for the question (RAG, module 4) and instruct the model to admit when the documentation doesn't mention something, citing sources.
2. Context window: the long conversation exceeds the window (or the history got truncated) and the model no longer "sees" its earliest answers. Mitigation: a history-management strategy — smart truncation or summarizing the previous conversation (03-03).
3. Knowledge cutoff: the new policy postdates the training (and it's internal, besides); the model falls back on what it saw in its data. Mitigation: supply the current version of the document in the context on every call, instead of relying on the model's knowledge.
4. Non-determinism: sampling produces variation across runs, and here the variation affects the substance (the citation). Mitigation: a low temperature for this use case and, above all, tests that verify properties (does it include the source?) run over several samples, with pass thresholds (06-03).

Solution 2:

• Cost per ticket ≈ 2,000/1M × €3 + 200/1M × €15 = €0.006 + €0.003 = €0.009.
• Monthly cost ≈ €0.009 × 3,000 = €27/month. Perfectly affordable — the quick math prevents both scares and unfounded vetoes: here the numbers back the feature.
• Design decisions that would reduce it (any of these counts): using a small model for summarizing (a simple task; it doesn't need the medium tier); summarizing only tickets over a certain length (short ones are their own summary); or trimming the input to the relevant parts of the ticket before the call.

Solution 3 (example):

"DocuBot generates its answers with artificial intelligence from Nubelia's internal documentation and may make mistakes or omit information. Always verify against the linked documentation any information you're going to use in production or communicate to a customer. Don't paste customer personal data or credentials into the chat. If you spot an incorrect answer, report it: it helps us improve the assistant."

(The essentials: acknowledged fallibility, verification of anything critical, a restriction on sensitive data; the feedback channel is a valuable extra.)

With this lesson you close the fundamentals module with a complete and honest view of the technology. You now know the seven limitations that condition any LLM design: hallucinations (the model optimizes for plausibility, not truth — like the naive DocuBot inventing Nubelia endpoints), the finite context window with no memory between calls, the knowledge cutoff that freezes what was learned, the non-determinism that forces you to test properties instead of exact text, the biases inherited from the data, the costs that scale with every token and every call, and the privacy of everything that travels to the provider. And you know the course's underlying answer: give the model the right context on every call — the pattern we'll develop as RAG in module 4 — together with systematic evaluation (module 6) and conscious design decisions. None of these limitations invalidates LLMs; all of them demand that the developer — you — design with them on the table. In the next module we move from understanding to acting: prompt engineering is the first concrete tool for steering the model's behavior, and we'll start with the anatomy of a prompt — roles, instructions and context — writing DocuBot's first real instructions.