The Project | About Us | Contribute | Donations | License
HOME My courses Completed
◄ Previous
◄

Probability Distributions

Next ►
►

Probability distributions are a fundamental concept in statistics and machine learning. They describe how the values of a random variable are distributed. Understanding probability distributions is crucial for data analysis, statistical inference, and building machine learning models.

Key Concepts

  1. Random Variable: A variable whose possible values are numerical outcomes of a random phenomenon.
  2. Probability Distribution: A function that describes the likelihood of obtaining the possible values that a random variable can take.
  3. Probability Density Function (PDF): For continuous random variables, the PDF describes the likelihood of the variable taking on a specific value.
  4. Probability Mass Function (PMF): For discrete random variables, the PMF describes the probability of the variable taking on a specific value.
  5. Cumulative Distribution Function (CDF): A function that describes the probability that a random variable will take a value less than or equal to a specific value.

Types of Probability Distributions

Discrete Distributions

  1. Bernoulli Distribution

    • Definition: Describes a random variable that has exactly two possible outcomes: success (1) and failure (0).
    • Example: Flipping a coin (Heads or Tails).
    • PMF: \( P(X = x) = p^x (1 - p)^{1 - x} \) where \( x \in {0, 1} \) and \( p \) is the probability of success.

  2. Binomial Distribution

    • Definition: Describes the number of successes in a fixed number of independent Bernoulli trials.
    • Example: Number of heads in 10 coin flips.
    • PMF: \( P(X = k) = \binom{n}{k} p^k (1 - p)^{n - k} \) where \( k \) is the number of successes, \( n \) is the number of trials, and \( p \) is the probability of success.

  3. Poisson Distribution

    • Definition: Describes the number of events occurring within a fixed interval of time or space.
    • Example: Number of emails received in an hour.
    • PMF: \( P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!} \) where \( \lambda \) is the average number of events per interval.

Continuous Distributions

  1. Normal Distribution

    • Definition: Describes a continuous random variable with a symmetric, bell-shaped distribution.
    • Example: Heights of people, test scores.
    • PDF: \( f(x) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x - \mu)^2}{2\sigma^2}} \) where \( \mu \) is the mean and \( \sigma \) is the standard deviation.

  2. Exponential Distribution

    • Definition: Describes the time between events in a Poisson process.
    • Example: Time until the next earthquake.
    • PDF: \( f(x) = \lambda e^{-\lambda x} \) for \( x \geq 0 \), where \( \lambda \) is the rate parameter.

  3. Uniform Distribution

    • Definition: Describes a random variable that has an equal probability of taking any value within a specified range.
    • Example: Rolling a fair die.
    • PDF: \( f(x) = \frac{1}{b - a} \) for \( a \leq x \leq b \), where \( a \) and \( b \) are the minimum and maximum values.

Practical Examples

Example 1: Binomial Distribution

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import binom

# Parameters
n = 10  # number of trials
p = 0.5  # probability of success

# Binomial distribution
x = np.arange(0, n+1)
pmf = binom.pmf(x, n, p)

# Plotting
plt.bar(x, pmf)
plt.xlabel('Number of Successes')
plt.ylabel('Probability')
plt.title('Binomial Distribution (n=10, p=0.5)')
plt.show()

Example 2: Normal Distribution

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm

# Parameters
mu = 0  # mean
sigma = 1  # standard deviation

# Normal distribution
x = np.linspace(-5, 5, 1000)
pdf = norm.pdf(x, mu, sigma)

# Plotting
plt.plot(x, pdf)
plt.xlabel('Value')
plt.ylabel('Probability Density')
plt.title('Normal Distribution (mu=0, sigma=1)')
plt.show()

Exercises

Exercise 1: Poisson Distribution

Task: Plot the PMF of a Poisson distribution with \( \lambda = 3 \).

Solution:

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import poisson

# Parameters
lambda_ = 3  # rate parameter

# Poisson distribution
x = np.arange(0, 15)
pmf = poisson.pmf(x, lambda_)

# Plotting
plt.bar(x, pmf)
plt.xlabel('Number of Events')
plt.ylabel('Probability')
plt.title('Poisson Distribution (lambda=3)')
plt.show()

Exercise 2: Exponential Distribution

Task: Plot the PDF of an exponential distribution with \( \lambda = 2 \).

Solution:

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import expon

# Parameters
lambda_ = 2  # rate parameter

# Exponential distribution
x = np.linspace(0, 3, 1000)
pdf = expon.pdf(x, scale=1/lambda_)

# Plotting
plt.plot(x, pdf)
plt.xlabel('Time')
plt.ylabel('Probability Density')
plt.title('Exponential Distribution (lambda=2)')
plt.show()

Summary

In this section, we covered the fundamental concepts of probability distributions, including the difference between discrete and continuous distributions. We explored several common distributions such as Bernoulli, Binomial, Poisson, Normal, Exponential, and Uniform distributions. Practical examples and exercises were provided to reinforce the concepts. Understanding these distributions is crucial for statistical analysis and building robust machine learning models.

◄ Previous
◄
Next ►
►
The Project | About Us | Contribute | Donations | License
© Copyright 2024. All rights reserved