In 02-05 we built a fraud detector with Bayes' theorem: we started from a prior (what fraction of orders are fraudulent), observed a piece of evidence and updated toward the posterior. That belief-updating machine was, without knowing it, a half-built classifier: it lacked the ability to combine several pieces of evidence at once without the arithmetic exploding. Naive Bayes completes the construction with one brazenly false assumption — that all features are independent of one another — which, against all odds, produces one of the fastest, most frugal and most useful classifiers in practice. In this lesson you'll make the full journey: from the theorem to the classifier, a numeric example by hand with MercaFresh orders, the three variants and when to use each, and the scikit-learn implementation.

1. From Bayes' theorem to the classifier
2. The "naive" assumption and why it works despite being false
3. Numeric example by hand: is this order fraudulent?
4. Laplace smoothing
5. The three variants: Gaussian, Multinomial, Bernoulli
6. Implementation with scikit-learn
7. Strengths and weaknesses

An express reminder of 02-05, in classification vocabulary:

$$P(\text{class} \mid \text{data}) = \frac{P(\text{data} \mid \text{class}) \cdot P(\text{class})}{P(\text{data})}$$

The Bayesian classifier is direct: compute each class's posterior and pick the one with the maximum posterior. Since $P(\text{data})$ is identical for all classes, we don't even need to compute it — comparing $P(\text{data} \mid \text{class}) \cdot P(\text{class})$ is enough.

Note the difference in philosophy from logistic regression (04-02): that model learned $P(\text{class} \mid \mathbf{x})$ directly, by fitting weights; Naive Bayes builds it from pieces — it learns how the data is distributed within each class and lets the theorem do the synthesis. That's why logistic regression is called a discriminative model and Naive Bayes a generative one.

The obstacle: with several features, the likelihood is a joint probability — $P(x_1, x_2, \dots, x_n \mid \text{class})$. Estimating it honestly requires having seen every combination of values many times, and the combinations grow exponentially: with 15 features it's impossible (another face of the curse of dimensionality from 04-05).

The "naive" way out: assume that, within each class, the features are independent of one another. Then the joint factorizes into a product of individual terms, each trivial to estimate:

$$P(x_1, \dots, x_n \mid c) \approx P(x_1 \mid c) \cdot P(x_2 \mid c) \cdots P(x_n \mid c)$$

The assumption is almost always false — at MercaFresh, an order's amount and its number of items are clearly correlated (02-03). Why does it work then? Two reasons:

• Classifying only requires ranking correctly. We don't need the posterior to be exact, only for the right class to end up above the others. Independence violations distort the probabilities (usually making them too extreme: correlated evidence gets "counted twice"), but they often distort all classes in similar directions, and the ranking survives.
• Fewer parameters, less variance. Estimating n univariate distributions per class requires little data and produces stable estimates; a model that respected every dependency would need far more data to avoid hallucinating. With scarce data, the biased-but-stable model beats the faithful-but-shaky one — a preview of the bias-variance trade-off of 06-05.

Practical consequence: trust Naive Bayes's labels, and handle its probabilities with tweezers (they tend toward unjustified 0.99s and 0.01s).

A history of 1,000 MercaFresh orders: 20 fraudulent (a 2% prior) and 980 legitimate. Three binary features per order, with their frequencies in each class:

An order arrives with high_amount = yes, new_address = yes, early_morning = no. Score for each class (prior × product of likelihoods, using $P(\text{no}) = 1 - P(\text{yes})$):

• Fraud: $0.02 \times 0.70 \times 0.80 \times (1 - 0.50) = 0.02 \times 0.28 = 0.0056$
• Legitimate: $0.98 \times 0.10 \times 0.15 \times (1 - 0.05) = 0.98 \times 0.01425 = 0.01397$

Legitimate wins (0.01397 > 0.0056). Normalizing to get the posterior: $P(\text{fraud}) = \frac{0.0056}{0.0056 + 0.01397} \approx 0.29$.

Two valuable readings:

• The prior weighs enormously: despite two alarm signals out of three, the 2% starting point keeps fraud below 50%. It's the same false-positive lesson from 02-05: with rare events, the evidence has to be overwhelming. Even so, a posterior of 29% against the 2% baseline — the order is 14 times more suspicious than average — amply justifies a manual review.
• Every feature multiplies: the absence of early_morning favored "legitimate" (0.95 against 0.50). In Naive Bayes no piece of evidence is neutral: they all push.

This is the entire algorithm. Training = counting frequencies (or means and variances). Predicting = multiplying and comparing. Hence its unbeatable speed.

A fatal flaw of pure counting: if no fraud in the history ever happened in the early_morning window, then $P(\text{early_morning} \mid \text{fraud}) = 0$, and that zero annihilates the entire product — no early-morning order could ever be classified as fraud, however scandalous its other signals. A single gap in the data would veto a whole class.

Laplace smoothing avoids it by adding a small fictitious count $\alpha$ (typically 1) to each cell: instead of $\frac{\text{cases}}{\text{total}}$, one estimates $\frac{\text{cases} + \alpha}{\text{total} + \alpha \cdot k}$ (with k possible values). That way no estimated probability is exactly 0 or 1 — "not seen yet" is treated as "very rare", not as "impossible". In sklearn it's the alpha parameter of the counting variants.

The only thing that changes between variants is how $P(x_i \mid c)$ is modeled according to the feature type:

GaussianNB estimates, for each feature and each class, the mean and deviation of a Gaussian bell — the normal distribution of 02-02 working as a likelihood — and evaluates the density of the observed value under each bell. MultinomialNB is the historical king of text classification (spam, sentiment): each document is represented as word counts, and each word's likelihood in each class is estimated with Laplace; you'll meet it again in the sentiment analysis project (09-03). If your features are mixed, the pragmatic route is to discretize/binarize toward one variant, or split and combine — although on mixed tabular data trees (04-03) usually perform better.

GaussianNB on MercaFresh's churn, with the usual Pipeline pattern. A preprocessing nuance: Naive Bayes uses no distances, so scaling is indifferent to it (like the tree, 03-05); on the other hand the Yeo-Johnson transformation from 03-03 suits it especially well, because it nudges each feature toward the normality GaussianNB presupposes:

Reading points:

• class_prior_ holds the priors estimated from train — each class's proportion, the "2%" of our hand example.
• theta_ and var_ store the mean and variance of each feature in each class: the whole model fits in two small tables. Compare with K-NN (04-05), which carries the entire dataset: they are the two extremes of the memory/synthesis spectrum.
• Speed: training is one pass of counts and moments — Naive Bayes trains in milliseconds where the SVM (04-04) takes minutes. That's why it's the go-to classifier for instant baselines and for systems with continuous retraining.
• For text, the same skeleton with MultinomialNB(alpha=1.0) after a count vectorization — the full recipe, in 09-03.

• Taking predict_proba literally. A 0.998 from Naive Bayes doesn't mean a real 99.8%: correlated evidence got counted several times. For deciding labels and ranking risks it's fine; for communicating probabilities to the business, not without calibration.
• Using MultinomialNB with continuous or negative features. It expects counts; with the output of the RobustScaler (negative values) it fails outright. Each variant with its feature type — the table in section 5 is the map.
• Forgetting smoothing with rare categories. With alpha=0, a category never seen in a class vetoes that class forever. Keep the default alpha=1 unless you have a solid reason.
• Dismissing it as "too simple". On text and as a baseline it remains competitive, and as a project's first model it delivers, in seconds, the reference the others must beat.
• Tip: when GaussianNB performs badly, draw the per-class histogram of your features (02-01). If you see bimodalities or brutal skews, the bell curve the model assumes looks nothing like reality — and the transformation from 03-03 or binning can recover several points of accuracy.

Exercise 1. Using the tables from the hand example, classify the order (high_amount = no, new_address = yes, early_morning = yes) and compute its normalized fraud posterior.

Exercise 2. In the history, no fraudulent order ever used cash_on_delivery (0 out of 20). Without smoothing, what fraud posterior would an order with cash on delivery, high amount, new address and early-morning timing have? Recompute $P(\text{cash_on_delivery} \mid \text{fraud})$ with Laplace ($\alpha = 1$, binary feature) and explain the qualitative change.

Exercise 3. On the churn data, train GaussianNB, the logistic regression (04-02) and the tree from 04-03, timing each one's fit (time.perf_counter). Compare accuracy and time, and reason through a business scenario where you'd choose Naive Bayes even though it's not the most accurate.

Exercise 1

• Fraud: $0.02 \times (1-0.70) \times 0.80 \times 0.50 = 0.02 \times 0.12 = 0.0024$
• Legitimate: $0.98 \times (1-0.10) \times 0.15 \times 0.05 = 0.98 \times 0.00675 = 0.006615$

Legitimate wins; fraud posterior $= \frac{0.0024}{0.0024 + 0.006615} \approx 0.27$. Curious: two alarm signals (new address and early morning) against one reassuring signal (normal amount) leave the order in the moderately suspicious zone — the 2% prior acting as an anchor again, as in 02-05.

Exercise 2

Without smoothing, $P(\text{cash_on_delivery} \mid \text{fraud}) = 0/20 = 0$, and the fraud score is $0.02 \times 0 \times \dots = 0$: a fraud posterior of exactly 0, however alarming the other three features are. The zero acts as an absolute veto. With Laplace: $\frac{0 + 1}{20 + 1 \cdot 2} = \frac{1}{22} \approx 0.045$. The fraud score comes back to life: $0.02 \times 0.045 \times 0.70 \times 0.80 \times 0.50 \approx 2.5 \times 10^{-4}$, now comparable with the legitimate score (which also incorporates its own cash-on-delivery term). The qualitative change: "never seen" goes from impossible to very unlikely, and the decision once again depends on the evidence as a whole rather than a single veto.

Exercise 3

Typical result: Naive Bayes is the fastest by far and lands close to (somewhat below) the others in accuracy. Scenarios where it would win the job anyway: (1) very frequent or incremental retraining — e.g. a filter that updates with each batch of orders using partial_fit; (2) very little labeled data — MercaFresh's confirmed fraud may be a handful of cases, and NB is one of the few things that doesn't overfit there; (3) as an instant baseline that sets the bar before investing in expensive models. Accuracy is not the only axis: cost, latency and model freshness decide too (we'll return to this in 08-02).

You've closed the circle opened in 02-05: Bayes' theorem, with the naive independence assumption as glue, becomes a complete classifier — priors by counting, likelihoods per feature (Gaussian bells, Laplace-smoothed frequencies or Bernoullis depending on the variant), and the maximum posterior as the decision. You can do the arithmetic by hand, you know why the model works even though its assumption is a lie, and you know where it belongs: the fastest, most frugal classifier in the module, ideal as a baseline and for text, with probabilities to be viewed skeptically.

With this you have six supervised algorithms, each with a distinct central idea: fitting lines, bending probabilities, chaining questions, maximizing margins, voting among neighbors, multiplying evidence. The module's final lesson introduces the family that can learn whatever idea is needed: small computing units — each suspiciously similar to logistic regression — connected in layers that compose arbitrarily complex functions. Neural networks arrive, and with them the module's close and final comparison.