The expert system of 06-02 decides returns with crisp rules because the policy is crisp: 14 days are 14 days. The incident diagnosis of case 9 is another story. A package that arrives dented suggests transport damage, but it may also be a picking error with poor packaging; a delay points to the Getafe warehouse, but the courier is late too; and incidents.csv has half its causes blank. Here IF-THEN rules break down: every symptom is compatible with several causes, with different strengths. This lesson teaches you to reason with degrees of belief. We will see why crisp rules are not enough; the two historical approaches that tried it inside expert systems (MYCIN's certainty factors and fuzzy logic) and why probability ended up prevailing; a review of probability from scratch (joint, conditional, independence, product rule, marginalisation) up to Bayes' theorem, with NovaMarket's diagnosis worked out by hand and in code; the Naive Bayes of 04-04 done by hand; Bayesian networks, with an incident diagnosis network and inference by enumeration in pure Python; and how to turn those probabilities into decisions with the expected utility of 02-01. It matters because almost all of modern AI, from the spam filter to the LLM, is applied probability; and because at NovaMarket Diego's question is not "what is the cause?" but "what do I do with this incident, knowing what I know?".

1. Why crisp rules are not enough
2. Historical approaches: certainty factors and fuzzy logic
3. Probability from scratch: joint, conditional, independence, product and marginalisation
4. Bayes' theorem, by hand and in code
5. Naive Bayes by hand: the cause of an incident from several symptoms
6. Bayesian networks: nodes, arcs and conditional probability tables
7. Code: incident diagnosis network and inference by enumeration
8. Approximate inference and learning the CPTs
9. Decisions under uncertainty: expected utility
10. Common Mistakes and Tips
11. Exercises
12. Conclusion

Let us try to write incident diagnosis as rules from 06-02. The incident types Diego records are: damaged package, product not delivered, wrong product, defective product and delay; the causes: picking error in the warehouse, transport damage, wrong address, stale stock (especially at Getafe, whose inventory syncs late) and supplier failure. A first attempt:

• IF damaged_package THEN cause = transport_damage
• IF delay THEN cause = stale_stock

And immediately the counterexamples: packages dented because of packaging badly chosen at picking; delays caused by wrong addresses that force a second attempt; wrong products that arrive that way from the supplier. Every rule is "almost always" true, and "almost always" does not fit into the logic of 06-01. Three underlying reasons:

• Laziness: enumerating every condition and exception so that a rule is exact is unfeasible (you would have to include the operator, the packaging, the route, the courier...).
• Theoretical ignorance: there is no complete theory saying what causes each symptom.
• Practical ignorance: even if there were, we do not have all the data of the case (we do not see the package until the customer photographs it).

What we do know is that a dented package makes transport damage more probable, and by how much. We need a language for degrees of belief that combine correctly when several pieces of evidence arrive and that can be estimated from incidents.csv. That language is probability; before we get there, let us look at the two shortcuts that were tried.

MYCIN (01-01, 06-02) attached to every rule a certainty factor CF between −1 (certain it is false) and +1 (certain it is true), and combined those of several rules supporting the same conclusion with a simple formula:

• Both positive: CF = CF₁ + CF₂·(1 − CF₁)
• Both negative: CF = CF₁ + CF₂·(1 + CF₁)
• Different signs: CF = (CF₁ + CF₂) / (1 − min(|CF₁|, |CF₂|))

Output:

Two pieces of evidence in favour reinforce each other (0.6 and 0.3 give 0.72, never more than 1); one against subtracts. It was intuitive, cheap and it worked in MYCIN. It was abandoned because it has no clear semantics: the numbers are neither probabilities nor anything well defined, the formula treats all evidence as independent (two symptoms that in reality always go together count twice), it does not distinguish "I don't know" from "contradictory evidence", and Heckerman showed in 1986 that CFs are only consistent with probability under very restrictive assumptions. When Bayesian networks made probability tractable, CFs became a museum piece.

Fuzzy logic (Zadeh, 1965) attacks a different problem: the vagueness of words. "Severe delay" is not true or false from an exact hour onwards; a 36-hour delay is "fairly mild" and "a bit severe". Each term has a membership function between 0 and 1:

Output:

On top of these memberships one writes fuzzy rules ("IF delay is severe AND customer is valuable THEN compensation is high") that are combined with minimums and maximums and defuzzified into a number. It is useful in control (washing machines, ABS brakes, air conditioning) and in interfaces where linguistic vagueness matters. But careful: the 0.5 membership of "severe" at 60 hours is not a probability; the delay is exactly 60 hours, with no uncertainty; what is fuzzy is the word. The uncertainty of the diagnosis ("was it transport or picking?") is something else: we do not know what happened, and for that the right instrument is probability.

We will work with discrete random variables: Cause (five values) and binary symptoms such as Dented. A distribution assigns to each value a number between 0 and 1, summing to 1. The figures that follow are fictional but plausible, rounded from NovaMarket's ~200 labelled incidents.

Prior probability (or marginal): what we believe before seeing the case. P(Cause = transport_damage) = 0.25 means that, of all incidents, one in four is due to transport.

Joint probability P(Cause, Dented): the probability of both things happening at once. The most intuitive way to see it is a frequency table over 1,000 incidents:

Each cell divided by 1,000 is the joint: P(transport_damage, Dented) = 0.20.

Marginalisation: summing the joint over the variable we are not interested in. P(Dented) = 30 + 200 + 3 + 4 + 15 = 252 out of 1,000 = 0.252. It is "the column total". As a formula: P(A) = Σ_c P(A, c).

Conditional probability P(A | B): the probability of A knowing that B happened: we restrict the table to B's row or column. P(Dented | transport_damage) = 200/250 = 0.80: of the transport damages, 80 % arrive dented. And P(transport_damage | Dented) = 200/252 = 0.794: of the dented ones, 79 % are transport damage. They are not the same thing: the first is a likelihood (what the cause produces), the second is what we want to diagnose. Confusing them is the most common error in this whole field.

Product rule: P(A, B) = P(A | B) · P(B). With our figures: 0.80 · 0.25 = 0.20. It is the definition of the conditional rearranged, and chained it gives the chain rule P(A, B, C) = P(A | B, C) · P(B | C) · P(C), which we will use in Bayesian networks.

Independence: A and B are independent if P(A | B) = P(A), that is, knowing B does not change what we believe about A; then P(A, B) = P(A)·P(B). Dented and Cause are not independent (0.80 versus 0.252). And there is a finer notion that will be the key to section 6: conditional independence: two symptoms may depend on each other (dented packages come with a wrong product more often than non-dented ones, because both point to handling errors) and yet be independent once the cause is known: if I already know it was a picking error, knowing it is dented tells me nothing new about whether the product is the wrong one. P(A | B, Cause) = P(A | Cause).

From the product rule written both ways, P(C, A) = P(A | C)·P(C) = P(C | A)·P(A), we solve:

P(C | A) = P(A | C) · P(C) / P(A)

• P(C): prior, what we believed about the cause before seeing the symptom.
• P(A | C): likelihood, how much that cause produces that symptom.
• P(A): evidence, the total probability of the symptom, computed by marginalising: P(A) = Σ_c P(A | c)·P(c). It makes the posteriors sum to 1.
• P(C | A): posterior, what we believe afterwards.

It is the formula that underpinned Naive Bayes in 04-04 and that we can now justify. We compute by hand P(Cause | Dented) for the five causes:

And in code, with fractions so that there is no rounding:

Output:

Note the role of the prior: supplier failure produces dented packages 15 % of the time, more than picking error (10 %), and yet picking has a higher posterior (0.119 versus 0.060) because it is three times more frequent. Ignoring the prior (looking only at "which cause best explains the symptom") is the base rate fallacy.

Link with 04-05. If we treat "dented" as a diagnostic test for "transport damage", the sensitivity of the test is P(Dented | transport) = 0.80 and its specificity is P(Not dented | not transport) = 1 − 52/750 = 0.931 (of the 750 incidents that are not transport, 52 are dented). Bayes in that language: posterior = sens·prev / (sens·prev + (1 − spec)·(1 − prev)) = 0.80·0.25 / (0.80·0.25 + 0.069·0.75) = 0.794. The positive predictive value of 04-05 is a Bayesian posterior, and it depends on the prevalence (the prior) as much as on the quality of the test: the same test applied at Getafe, where transport weighs less, would give a lower posterior.

With one symptom a table is enough. With three (damaged package, wrong product, delay) the joint P(Cause, S₁, S₂, S₃) would have 5·2·2·2 = 40 cells to estimate; with twenty symptoms, millions. Naive Bayes (04-04) gets by with the conditional independence assumption of section 3: given the cause, the symptoms are independent, so P(S₁, S₂, S₃ | c) = P(S₁ | c)·P(S₂ | c)·P(S₃ | c) and only three small tables are needed:

Incident 7731: damaged package yes, delay yes, wrong product no. For each cause we multiply the prior by the three likelihoods (for "no" we use 1 − p) and normalise:

Output:

It is exactly what scikit-learn's GaussianNB or MultinomialNB do internally (with the likelihoods estimated from the data, section 8). "Delay yes" alone, without a damaged package, would give stale_stock 0.51 and wrong_address 0.28: every new symptom moves the belief, and the product of likelihoods is the correct way of combining evidence that certainty factors lacked.

Naive Bayes is a very particular Bayesian network: one cause with arrows to every symptom. A general Bayesian network is a directed acyclic graph in which:

• every node is a random variable;
• every arc X → Y expresses a direct (often causal) influence of X on Y;
• every node carries a conditional probability table (CPT) giving P(node | parents): for a node with no parents, its prior; for one with parents, one row per combination of the parents' values.

The network encodes conditional independences: each node is independent of its non-descendants given the information of its parents. Thanks to that, the full joint factorises via the chain rule into the product of the CPTs: P(X₁, ..., Xₙ) = Π P(Xᵢ | parents(Xᵢ)). Instead of one giant table, small local tables that an expert (Diego) can fill in or that are estimated from data.

Our incident diagnosis network adds to Naive Bayes a parent node, the warehouse, because the distribution of causes is not the same at Zaragoza as at Getafe (there, stale stock weighs much more), and a fourth symptom, "not delivered":

CPT of the cause node (one row per warehouse; each row sums to 1):

CPTs of the four symptoms (P(symptom = yes | cause); the first three rows are those of section 5):

Count parameters: 1 + 8 + 4·5 = 29 numbers versus the 2·5·2⁴ − 1 = 159 of the full joint. And read the independences: the symptoms are conditionally independent of each other given the cause (which is why Naive Bayes is a special case), and the warehouse influences the symptoms only through the cause: P(delay | cause, warehouse) = P(delay | cause).

Inference means computing P(query | evidence) for any node and any evidence. The simplest algorithm is enumeration: the joint is obtained by multiplying the CPTs, and to obtain P(query, evidence) we sum the joint over all combinations of the hidden variables (neither queried nor observed); dividing by the total sum normalises. It is Bayes and marginalisation, nothing more.

Output:

Explanation:

• NETWORK is the whole network: for each node, its parents, its possible values and its CPT as a dictionary of dictionaries; binary_cpt saves writing every symptom twice. ORDER lists the nodes with parents before children, so that the chain rule can be applied.
• local_prob reads the CPT: it looks up the row for the current values of the parents and returns the probability of the node's value. joint_prob multiplies the six local probabilities: P(warehouse)·P(cause | warehouse)·P(damaged | cause)·... It is the factorisation of section 6.
• infer does the enumeration: for each value of the query, it sums the joint over all combinations of the hidden variables (product), which gives P(query = value, evidence); at the end it divides by the sum (P(evidence)) to obtain the posterior. With 160 combinations in total it is instantaneous.
• Read the answers as a diagnostician would: with no evidence, the most probable cause is picking (0.31); a delay alone shifts belief towards stale stock (0.375) and wrong address; knowing that the order left from Getafe pushes stale stock up to 0.567 (the warehouse influences the symptom through the cause); the full evidence of incident 7731 (damaged, delayed, correct product, delivered) gives transport 0.864, a little more than the 0.816 of Naive Bayes because "delivered" rules out wrong address and stale stock. The network also reasons upwards: a wrong product makes Zaragoza more probable (0.646 versus the prior 0.60), because picking weighs more there; and between symptoms: knowing the warehouse and the delay, it predicts whether the package will arrive (0.397 that it will not). All with the same 29 figures.

This last point is what a rule system cannot do: use the same knowledge to diagnose (symptoms → cause), predict (cause → symptoms) and explain (given one symptom, which other one to expect?). And the observed variables are explained with the trace of the formula: "transport 0.864 because the prior of transport across Zaragoza/Getafe is 0.26 and a damaged package is 8 times more likely with transport than with picking".

Two notes so as not to create false expectations:

• Approximate inference. Enumeration sums over every combination of hidden variables: exponential. With six nodes that is 160; with fifty symptoms and intermediate causes, intractable. The smarter exact algorithms (variable elimination, junction trees) exploit the structure of the graph, and when even that is not enough, sampling is used: generate thousands of "virtual incidents" following the CPTs (direct sampling, rejection sampling, likelihood weighting, MCMC) and count in what fraction the cause is each thing. It is the same Monte Carlo idea of module 3 and of AlphaGo's search.
• Learning the CPTs from incidents.csv. The 29 figures of the network were set by us; in practice they are estimated by counting: P(damaged_package = yes | cause = transport) ≈ (number of transport incidents with a damaged package + 1) / (number of transport incidents + 2), with the "+1/+2" of Laplace smoothing so that a symptom never seen with a cause does not get a probability of exactly zero (which would wipe out any product). It is what MultinomialNB.fit does. And here the problem of 02-03 reappears: only ~200 rows have cause; with 5 causes and 4 symptoms, some cells are estimated from fewer than ten cases, and the 200 were filled in on quiet days (sampling bias). Diego was right to make the cause mandatory: the quality of the network is set by the data its tables are estimated from; the structure (which arrow goes where) is usually provided by the expert, although algorithms to learn it also exist.

Knowing that P(transport | evidence) = 0.864 is not yet an action. In 02-01 we said that the rational agent chooses the action with the highest expected utility: for each action, sum over the possible states of the probability of the state times the utility of the outcome. Faced with a €60 incident, Diego has two actions: refund directly (fast, the customer is happy, but the chance to recover the amount from the carrier or the product is lost) or open an investigation (it costs a person's time and delays the resolution, but depending on the cause part of the cost is recovered). We put utilities in euros (negative: cost to NovaMarket), from operations experience:

Output:

Knowing nothing, investigating pays off (−45 versus −60). With a delay, still, but only just. With a delay and Getafe, stale stock is so probable (0.567) that investigating costs more than refunding: the decision changes with the evidence. With a damaged package, investigating is clearly better because a claim against the carrier is almost certain. This is a utility-based agent from 02-01 with the network as its model of the world, and it is also the scheme of the human review of 02-04: a third action, "hand over to a person", can be added with its cost, and expected utility left to say when it is worth it. The network diagnoses; utility decides; and both parts are readable and adjustable by Diego.

• Confusing P(symptom | cause) with P(cause | symptom). The first is given by the expert or the data ("80 % of transport damages arrive dented"); the second is the one you are after and it needs the prior. Bayes is the bridge; do not cross it in your head without it.
• Ignoring the base rate. The cause that "best explains" the symptom is not the most probable if it is rare. Always look at the prior column.
• Treating fuzzy memberships or certainty factors as probabilities. They are not; do not combine them with Bayes or sum over them.
• Zeros in the CPTs. A 0 estimated from little data annihilates the product and makes a cause impossible forever. Smooth (Laplace) and reserve 0 for what is logically impossible.
• Assuming conditional independence without thinking about it. Naive Bayes works surprisingly well, but if two symptoms are almost the same (delay and "not delivered on the date"), they will count twice. A network with the right structure corrects this.
• Forgetting that probabilities belong to a context. CPTs estimated from Zaragoza incidents are not valid for Getafe: that is why the warehouse is a node.
• Stopping at the probability. Diego's goal is to decide; without utilities there is no decision, and a badly set utility (forgetting the cost of an angry customer) decides badly even if the network is perfect.

Exercise 1. With the table of 1,000 incidents of section 3, compute by hand: (a) P(not dented); (b) P(picking_error | not dented); (c) P(transport_damage | not dented). Check that (b) is greater than the picking prior and (c) lower than the transport prior, and explain why in one sentence.

Exercise 2. With infer, compute P(cause | not_delivered = True) for warehouse = "zaragoza" and for warehouse = "getafe". Which is the most probable cause at each warehouse? Which action of section 9 would you choose in each case (compute the expected utilities with decide)? Explain the role of the warehouse node.

Exercise 3. A product arrives wrong and, in addition, the package is damaged. Compute P(cause | wrong_product) and P(cause | wrong_product, damaged_package). Does picking_error go up or down when the damage is added? And supplier_failure? Explain the result in terms of the likelihoods in the damaged_package CPT.

Solution 1. (a) 748/1,000 = 0.748. (b) 270/748 = 0.361 (prior 0.30). (c) 50/748 = 0.067 (prior 0.25). A package that is not dented is evidence against transport (which almost always dents) and, by elimination, in favour of the other causes: the absence of a symptom is informative too, and all the more so the more sensitive the symptom is to the cause.

Solution 2. infer("cause", {"not_delivered": True, "warehouse": "zaragoza"}) gives wrong_address 0.525, supplier_failure 0.148, stale_stock 0.132, transport 0.123, picking 0.072; with Getafe: stale_stock 0.445, wrong_address 0.344, supplier 0.101, transport 0.067, picking 0.042. At Zaragoza, the most probable cause of a "not delivered" is a wrong address; at Getafe, stale stock. decide({"not_delivered": True, "warehouse": "zaragoza"}) gives investigate −39.02 € versus −60: investigate, clearly (the address is corrected and the reshipment is charged); with Getafe, investigate −59.23 € versus −60: a practical tie, where a little more weight on stale stock (or a slightly higher investigation cost) would be enough for refunding to win. The warehouse node changes the prior of the causes, and with it the whole diagnosis and the decision, without touching the symptom CPTs.

Solution 3. P(cause | wrong): picking 0.789, supplier 0.122, stale_stock 0.065, transport 0.019, wrong_address 0.005. P(cause | wrong, damaged): picking 0.694, supplier 0.161, transport 0.133, stale_stock 0.012, wrong_address 0.001. Picking goes down (0.789 → 0.694) and supplier and transport go up: damage is unlikely with picking (0.10), somewhat more likely with supplier (0.15) and far more likely with transport (0.80), so the new evidence redistributes belief towards them; but picking remains the most probable cause because a wrong product is 35 times more likely with picking than with transport. Bayes does exactly the arithmetic that intuition does by eye.

In this lesson we learned to reason when rules are not enough. We saw why incident diagnosis does not fit into crisp IF-THENs (laziness, theoretical and practical ignorance), and we placed the two historical shortcuts: MYCIN's certainty factors, intuitive but without semantics, and fuzzy logic, which models the vagueness of words ("severe delay" with membership 0.5) but not uncertainty about facts. We built probability from the table of 1,000 incidents: joint, marginalisation, conditional, product rule and independence (and conditional independence), up to Bayes' theorem, which diagnosed transport damage with 0.794 for a dented package, and which we also read in the language of sensitivity and specificity of 04-05. We did Naive Bayes by hand with three symptoms (0.816 for transport in incident 7731) and generalised to a six-node Bayesian network (warehouse → cause → four symptoms, 29 parameters instead of 159) with inference by enumeration in pure Python, able to diagnose, predict and reason upwards with the same knowledge; we noted how inference is approximated when the network grows and how the CPTs are learned from incidents.csv with smoothing. And we closed the circle with 02-01: expected utility turns the posterior into the decision to investigate or refund, and the decision changes when the Getafe evidence arrives.

With this NovaMarket has the three symbolic tools of the module: logic to represent (06-01), the expert system to decide with rules (06-02) and probability to decide under uncertainty (06-03). What remains is to see where they are really used, today, outside our laboratory: in the medicine MYCIN inaugurated, in banking and regulatory compliance, in industry, in law and in e-commerce; what expert systems have become (business rule engines, decision tables, knowledge graphs) and, above all, how they combine with the ML and LLMs of modules 4 and 5 in the neurosymbolic approach we announced at the close of 05-05: a model that estimates a probability and a rule that decides with it. That is the topic of the last lesson of the module, 06-04.