The LLM ecosystem evolves at breakneck speed: the specific models topping the rankings today may be surpassed within a few months. That's why this lesson doesn't try to hand you "the best model" (any ranking you memorized would expire almost immediately), but something far more durable: a framework of criteria for evaluating and comparing models, a map of the main providers and families, and a decision methodology you can apply today and two years from now. At the end we'll apply it to the case at hand: choosing the model approach for DocuBot, Nubelia's documentation assistant.

1. The five criteria for comparing models
2. Landscape of providers and model families
3. Proprietary models via API vs. self-hosted open models
4. Families and sizes: there is no single model per provider
5. How to choose: the methodology applied to DocuBot

Whenever you evaluate any model — current or future — these are the five dimensions to look at. Internalize them: provider spec sheets practically read themselves once you know what to look for.

How well does the model handle the tasks you care about? Mind the nuance: the tasks you care about, not "in general". A model can be brilliant at reasoning about code and mediocre at writing in the other languages some of your users need, or vice versa.

• Providers publish results on standard benchmarks (knowledge exams, programming, reasoning). They're useful as a first filter, but they saturate (many models hover near the maximum) and they don't reflect your specific case.
• The only reliable evaluation is trying the model on your own cases: in Nubelia's case, with real support questions and real fragments of its documentation. In lesson 02-04 and in module 6 we'll see how to systematize this evaluation.

How much text (in tokens, as we saw in 01-02) the model can handle in a single call, input and output combined. It determines how much documentation, conversation history or instructions you can include. Current windows range from tens of thousands to millions of tokens depending on the model — but be careful: a huge window isn't free (more tokens = more cost and latency), nor does it guarantee the model will make good use of everything you pass it (we'll qualify this in 01-04).

Models consumed via API are billed per token, with different prices for input and output tokens (output is usually several times more expensive). The specific prices change often; what doesn't change is the structure:

Between the most capable and the most economical model from the same provider there can be one or two orders of magnitude of price difference. That's why the right question is never "which model is best?" but "which is the cheapest model that handles my task at the quality I need?". (Operational cost management — caching, optimization — is covered in detail in lesson 03-04.)

How long the model takes to respond. It's worth distinguishing two metrics:

• Time to first token: how long the user waits before seeing anything. Critical in conversational interfaces (which is why streaming exists — we'll see it in 03-02).
• Generation speed (tokens per second): how long the response takes to complete.

General rule: bigger, more capable models are slower and more expensive; smaller ones are faster and cheaper. For an interactive chat like DocuBot, latency matters a lot; for a nightly job that summarizes tickets, hardly at all.

Beyond text: does it accept images or PDFs as input (vision)? Audio? Does it support function calling (module 5) and structured outputs (02-03)? Does it offer extended reasoning modes for complex problems? Not every model offers everything, and if your case needs, say, to interpret screenshots attached to tickets, this becomes a knockout criterion.

Let's look at the main players in the ecosystem. The description is deliberately timeless: retain who's who and what characterizes each one, and check their websites for the details of the moment.

• Anthropic — the Claude family. Proprietary models accessible via API, with a strong focus on reliability, safety and assistant capabilities for working with code and long documents. The family is organized into capability/cost tiers (traditionally with names like Opus, Sonnet and Haiku, from largest to smallest).
• OpenAI — the GPT family (and derivatives). The provider that popularized conversational LLMs. Proprietary models via API, a broad range spanning from lightweight models to advanced reasoning models, and a very extensive tooling ecosystem.
• Google (DeepMind) — the Gemini family. Proprietary models integrated into the Google Cloud ecosystem, historically strong in multimodality (text, image, audio, video) and in very large context windows.
• Meta — the Llama family. The great driver of open-weights models: models you can download and run on your own infrastructure. They've spawned an enormous ecosystem of community variants and fine-tunes.
• Mistral AI. A European (French) company combining open models and commercial models via API. Also relevant for European data-sovereignty considerations.
• Other players worth knowing: DeepSeek and Qwen (Alibaba) as very competitive open families of Chinese origin; Cohere, enterprise-oriented; and the platform providers — AWS Bedrock, Google Vertex AI, Azure AI Foundry — which don't create the leading models but offer them within their cloud, with billing, security and regulatory compliance integrated into your existing infrastructure.

Three cross-cutting observations more useful than any ranking:

1. Every major provider offers a range, not one model. Each family has tiers (large/medium/small) with the same API style.
2. The quality frontier moves constantly and converges. Today's leader is caught up with in months. Design your application so you can switch models easily (a good abstraction over the LLM layer, and your own evaluations to re-run against a new model).
3. The APIs are conceptually very similar (messages with roles, parameters like temperature, streaming, tools), so what you learn in this course with one provider transfers almost directly to the others. We'll cover the practical integration details in module 3.

The most structural decision isn't "which provider" but which deployment model: do I consume a proprietary model as a service, or do I download an open model and run it on my infrastructure?

Important nuances the table doesn't capture:

• "Open" almost always means "open weights" (you can download and run the model), not that the training data or process are public. Always check the license: some allow unrestricted commercial use and others impose conditions.
• There's a very popular middle path: consuming open models through an inference provider that serves them via API. You get often-low prices and zero infrastructure, in exchange for your data leaving your perimeter after all. And the platform path (Bedrock, Vertex, Azure) offers proprietary models within your existing cloud and compliance framework.
• The cost of self-hosting is systematically underestimated. It's not just the GPU: it's availability, updates, inference optimization, observability and the staff who maintain it all.

An honest rule of thumb: for the vast majority of teams starting out, the proprietary (or platform) API is the right starting point — you iterate fast, with no upfront investment, at the best quality. Self-hosting is justified by strict data-residency requirements, very high sustained volumes, or deep customization needs.

This deserves its own section because it's a decision you'll make constantly: within your chosen provider, which tier of the range do you use?

• Large (frontier) model: maximum reasoning capability and reliability; more expensive and slower. For complex or ambiguous tasks, or where errors are costly.
• Medium model: the workhorse. The right quality/cost/latency balance for most applications — chatbots, RAG, document processing.
• Small model: very fast and cheap. For simple, high-volume tasks: classifying, extracting fields, routing requests.

And an architectural idea you'll see in mature systems: you don't have to pick just one. A single product can combine tiers depending on the subtask:

This complexity-based routing pattern reduces cost and latency without sacrificing quality where it matters. We'll come back to it when discussing costs (03-04).

Let's close with a five-step methodology, applying it to the decision Nubelia must make for DocuBot.

Step 1 — Define the use case's requirements. For DocuBot:

• Users: ~200 internal employees (developers and support). Moderate volume: maybe a few hundred questions a day.
• Task: answer questions in English (and occasionally other languages) based on technical documentation passed in as context.
• UX: interactive chat → perceived latency matters.
• Data: the internal documentation is confidential, but contains no customer personal data → an API with adequate contractual guarantees (no training on the data, limited retention) is acceptable for Nubelia; there's no hard self-hosting requirement.
• Errors are tolerable but annoying: a wrong answer wastes time, not money directly.

Step 2 — Filter by knockout criteria. It needs: a context window large enough for several documentation fragments plus the history, solid multilingual handling for the occasional non-English question, and (looking ahead to modules 4 and 5) good support for structured outputs and function calling. It doesn't need: vision, audio, a gigantic window.

Step 3 — Decide the deployment model. Small team, no experience serving models, moderate volume, no strict residency requirement → managed API. Reviewing the provider's data-processing terms is part of the decision, not an afterthought (we'll go deeper in 06-02).

Step 4 — Choose the tier, not just the provider. A reasonable initial hypothesis: a medium model from a major provider for answer generation. It's a hypothesis, not a commitment: the architecture must allow changing it.

Step 5 — Validate with your own evaluation and leave the door open. Before committing, the Nubelia team should prepare 30–50 real questions with known correct answers and test 2–3 candidate models against them. That evaluation set also becomes a permanent asset: every time a new model comes out, you re-run it and the migration decision is made with data. (How to build these evaluations is covered in 02-04 and 06-03.)

• Choosing a model based on headlines or generic rankings. Public benchmarks don't measure your task. A mid-tier model can beat a frontier one on your specific case (because of language, domain or format). Your own evaluation is irreplaceable.
• Starting with the biggest model "to be safe". Start by testing the medium tier; move up only if your evaluation shows you need to. The cost and latency difference is large, and many tasks don't justify it.
• Coupling yourself to a provider without an abstraction layer. If LLM calls are scattered across your code with provider-specific details, switching models becomes a project instead of a configuration change. Centralize LLM access from day one (we'll do it that way in module 3).
• Confusing "open model" with "free". The weights are free; the GPUs, the operations and the staff are not. Work out the total cost before deciding to self-host.
• Ignoring the license and the data terms. In open models (commercial-use restrictions) as much as in APIs (do they train on your data? how long do they retain it? where is it processed?). For a company like Nubelia, this is a first-order selection criterion.
• Tip: keep a small comparison sheet of your own (the 5 criteria × your 2–3 candidates) and update it whenever you evaluate new models. It's cheap, and it turns every re-evaluation into routine instead of debate.

For each scenario, indicate which of the five criteria (capability, context, cost, latency, modalities) would be the most decisive and why:

1. A nightly job that classifies 500,000 emails a month into 10 categories.
2. An assistant that analyzes lengthy contracts (hundreds of pages) and answers questions about them.
3. Real-time text autocompletion while the user types.
4. An internal tool that receives error screenshots and describes the problem.

A financial company wants an internal assistant that works with documents which, by strict internal policy, cannot leave its data center under any circumstances. It has a platform team experienced with Kubernetes and GPUs, and a high, sustained expected volume. Justify which deployment model you'd recommend and mention two hidden costs they should budget for.

At the DocuBot decision meeting, a Nubelia executive proposes: "Let's use the most powerful model on the market for everything; that way it's guaranteed to work". Write a reasoned reply (5–8 lines) using this lesson's methodology: what's wrong with the proposal and what you'd propose instead.

Solution 1:

1. Cost. Enormous volume and a simple task: the per-token price difference multiplied by 500,000 calls dominates the decision. A small model will probably classify just as well; the extra capability would be money down the drain.
2. Context. Hundreds of pages are a lot of tokens: you need a large window (or chunking techniques, as we'll see in module 4). If the document doesn't fit, the other criteria don't even apply.
3. Latency. Autocompletion must appear in fractions of a second or it gets in the way instead of helping. Aim for small, fast models.
4. Modalities. You need image input (vision); any model that doesn't support it is eliminated, no matter how cheap or capable.

Solution 2:

Recommendation: self-hosted open model. The requirement that data never leave the data center is a knockout for external APIs (even with contractual guarantees, the data would leave the perimeter), and the company has exactly the profile that makes self-hosting viable: an experienced platform team and a high, sustained volume that amortizes the infrastructure. Two hidden costs to budget for (other reasonable ones count too): (a) ongoing operations — model updates, inference optimization, high availability, observability — which require permanent staff dedication, not just the initial deployment; (b) the opportunity cost in quality: open models may lag the frontier, so they'll need continuous in-house evaluation and possibly fine-tuning to reach the target quality.

Solution 3 (example answer):

"Using the most powerful model for everything optimizes a single criterion (capability) and ignores the other four. We'd be paying the highest per-token price and the highest latency on the market for tasks where a medium model delivers the same quality: most questions to DocuBot are direct lookups against documentation we'll be passing in as context, not deep reasoning problems. I propose the opposite: define 30–50 real questions from our teams with their correct answers, evaluate a medium and a large model from 2–3 providers against them, and pick the cheapest model that reaches the target quality, keeping the architecture ready to switch models if a future evaluation justifies it. That way the decision rests on our own data, not generic rankings, and the evaluation set serves us forever."

In this lesson you've acquired what truly lasts in an ecosystem that changes every few months: the five comparison criteria (capability, context, cost, latency, modalities), the map of providers and families (Anthropic/Claude, OpenAI/GPT, Google/Gemini, Meta/Llama and the open ecosystem, Mistral and others, plus the cloud platforms), the proprietary API vs. self-hosting dilemma with its nuances, and a five-step methodology that we've applied to DocuBot: Nubelia's initial hypothesis is a medium model via managed API, validated with its own evaluation and with the architecture ready to switch. The underlying lesson: learn to evaluate, don't memorize rankings.

But before building anything, the other side of the coin remains. We've talked about what the models offer; in the next lesson, "Limitations and Risks: Hallucinations, Context and Costs", we'll look at what can go wrong: why DocuBot could confidently invent a Nubelia feature that doesn't exist, what the context window really implies, and what other risks — knowledge cutoff, non-determinism, biases, costs — you must keep on your radar from the design phase. It's the lesson that separates those who integrate LLMs with judgment from those who get surprises in production.