Bayes' Theorem is a fundamental concept in probability theory and statistics that describes the probability of an event based on prior knowledge of conditions related to the event. It is named after Thomas Bayes, an 18th-century statistician and minister. Bayes' Theorem is widely used in various fields, including machine learning, to update the probability estimates as more evidence or information becomes available.

Conditional probability is the probability of an event occurring given that another event has already occurred. It is denoted as \( P(A|B) \), which reads as "the probability of A given B."

The prior probability, denoted as \( P(A) \), is the initial probability of an event before any additional evidence is taken into account.

The posterior probability, denoted as \( P(A|B) \), is the updated probability of an event after considering new evidence.

The likelihood, denoted as \( P(B|A) \), is the probability of observing the evidence given that the event has occurred.

The marginal probability, denoted as \( P(B) \), is the total probability of the evidence under all possible scenarios.

Bayes' Theorem can be mathematically expressed as:

\[ P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)} \]

Where:

• \( P(A|B) \) is the posterior probability.
• \( P(B|A) \) is the likelihood.
• \( P(A) \) is the prior probability.
• \( P(B) \) is the marginal probability.

Let's consider a practical example to understand Bayes' Theorem better.

Suppose a medical test is used to detect a rare disease that affects 1 in 1,000 people. The test has a 99% sensitivity (true positive rate) and a 99% specificity (true negative rate). If a person tests positive, what is the probability that they actually have the disease?

1. Define the Events:

   • \( D \): The event that the person has the disease.
   • \( T \): The event that the person tests positive.

2. Given Probabilities:

   • \( P(D) = 0.001 \) (prior probability of having the disease)
   • \( P(T|D) = 0.99 \) (likelihood of testing positive given the disease)
   • \( P(T|\neg D) = 0.01 \) (likelihood of testing positive given no disease)
   • \( P(\neg D) = 0.999 \) (prior probability of not having the disease)

3. Calculate the Marginal Probability \( P(T) \): \[ P(T) = P(T|D) \cdot P(D) + P(T|\neg D) \cdot P(\neg D) \] \[ P(T) = (0.99 \cdot 0.001) + (0.01 \cdot 0.999) = 0.00099 + 0.00999 = 0.01098 \]

4. Apply Bayes' Theorem: \[ P(D|T) = \frac{P(T|D) \cdot P(D)}{P(T)} \] \[ P(D|T) = \frac{0.99 \cdot 0.001}{0.01098} \approx 0.0902 \]

So, the probability that a person actually has the disease given that they tested positive is approximately 9.02%.

A factory produces 1% defective items. A quality control test correctly identifies defective items 95% of the time and correctly identifies non-defective items 90% of the time. If an item tests positive for being defective, what is the probability that it is actually defective?

Solution:

1. Define the events:

   • \( D \): The event that the item is defective.
   • \( T \): The event that the item tests positive.

2. Given probabilities:

   • \( P(D) = 0.01 \)
   • \( P(T|D) = 0.95 \)
   • \( P(T|\neg D) = 0.10 \)
   • \( P(\neg D) = 0.99 \)

3. Calculate the marginal probability \( P(T) \): \[ P(T) = P(T|D) \cdot P(D) + P(T|\neg D) \cdot P(\neg D) \] \[ P(T) = (0.95 \cdot 0.01) + (0.10 \cdot 0.99) = 0.0095 + 0.099 = 0.1085 \]

4. Apply Bayes' Theorem: \[ P(D|T) = \frac{P(T|D) \cdot P(D)}{P(T)} \] \[ P(D|T) = \frac{0.95 \cdot 0.01}{0.1085} \approx 0.0876 \]

So, the probability that an item is actually defective given that it tested positive is approximately 8.76%.

A spam filter is 99% accurate in identifying spam emails and 98% accurate in identifying non-spam emails. If 5% of all emails are spam, what is the probability that an email is spam given that it was identified as spam by the filter?

Solution:

1. Define the events:

   • \( S \): The event that the email is spam.
   • \( T \): The event that the email is identified as spam.

2. Given probabilities:

   • \( P(S) = 0.05 \)
   • \( P(T|S) = 0.99 \)
   • \( P(T|\neg S) = 0.02 \)
   • \( P(\neg S) = 0.95 \)

3. Calculate the marginal probability \( P(T) \): \[ P(T) = P(T|S) \cdot P(S) + P(T|\neg S) \cdot P(\neg S) \] \[ P(T) = (0.99 \cdot 0.05) + (0.02 \cdot 0.95) = 0.0495 + 0.019 = 0.0685 \]

4. Apply Bayes' Theorem: \[ P(S|T) = \frac{P(T|S) \cdot P(S)}{P(T)} \] \[ P(S|T) = \frac{0.99 \cdot 0.05}{0.0685} \approx 0.7226 \]

So, the probability that an email is actually spam given that it was identified as spam by the filter is approximately 72.26%.

Bayes' Theorem is a powerful tool for updating probabilities based on new evidence. It is widely used in various applications, including medical diagnosis, spam filtering, and machine learning. Understanding and applying Bayes' Theorem can significantly enhance your ability to make informed decisions based on probabilistic reasoning.