Lecture 10: Foundations of Uncertainty & Probability | SE444

Back to Course Overview

Lecture Overview

This lecture introduces a fundamental paradigm shift in AI: moving from formal logic's absolute certainty to probabilistic reasoning under uncertainty. We explore why real-world AI systems need probability theory, covering Bayesian reasoning, conditional probability, and the philosophical contrast between two views of intelligence.

Key Concepts: Probability Theory, Bayes' Rule, Conditional Independence, Bayesian Inference

Applications: Medical Diagnosis, Robotics, NLP, Computer Vision, Decision Making

Two Perspectives on Intelligence

Historically, humanity has viewed intelligence from two fundamentally different perspectives:

Perspective 1: Absolute Knowledge (Logic-Based)

Intelligence as deductive reasoning from certain facts
Propositional and first-order logic
Rule-based expert systems
Assumption: Complete and certain knowledge

Perspective 2: Partial Knowledge (Probabilistic)

Intelligence as reasoning under uncertainty
Probability as degrees of belief
Learning from incomplete, noisy data
Assumption: Uncertainty is fundamental to intelligence

"From proving truths to estimating truths" - This lecture explores why modern AI embraces uncertainty not as a weakness, but as a more realistic model of human intelligence and the real world.

📚 Probability Cheat Sheet Available!

Quick reference guide covering probability axioms, Bayes' rule, distributions, and key formulas. Perfect for studying and quick lookups!

View Cheat Sheet

📊 Lecture 10 Presentation Slides

Complete lecture presentation (32 slides) covering reasoning under uncertainty, probability theory, Bayes' Rule, and Bayesian inference. Perfect for reviewing lecture content!

Main Topics

1. Two Perspectives on Intelligence

Historical evolution of AI: from rule-based logic to probabilistic reasoning. Why the shift from absolute to partial knowledge represents a paradigm change in AI.

AI History Logic vs Probability Paradigm Shift Philosophy of AI

2. Limits of Formal Logic in Real Worlds

Understanding why deterministic logic fails in open, noisy, and incomplete domains. Real-world examples: noisy sensors, incomplete databases, contradictory evidence.

Noisy Data Incomplete Knowledge Real-World Challenges Contradictions

3. Uncertainty as a New Paradigm

Belief states, partial observability, and rational decision-making under ignorance. How uncertainty modeling enables more robust and realistic AI systems.

Belief States Partial Observability Rational Decisions Uncertainty Modeling

4. Foundations of Probability Theory

Events, sample spaces, random variables, and the three axioms of probability. Understanding probability as both frequency and degree of belief.

Sample Spaces Random Variables Axioms Frequentist vs Bayesian

5. Conditional Probability & Bayes' Rule

Derivation and intuition behind Bayes' theorem. Classic examples: medical diagnosis, burglary-earthquake-alarm scenario, and Bayesian inference fundamentals.

Conditional Probability Bayes' Theorem Prior & Posterior Classic Examples

6. Joint, Marginal & Conditional Distributions

Understanding probability distributions over multiple variables. Visualization using tables and graphs. Marginalization and conditioning operations.

Joint Distributions Marginalization Conditioning Probability Tables

7. Independence & Conditional Independence

How independence assumptions simplify probability models. The crucial concept of conditional independence and its role in efficient inference.

Independence Conditional Independence Model Simplification d-separation

8. Statistical Foundations: LLN & CLT

Law of Large Numbers and Central Limit Theorem. Understanding convergence, sampling distributions, and why these theorems matter for machine learning.

Law of Large Numbers Central Limit Theorem Convergence Statistical Foundations

9. Bayesian Inference in Practice

Working through complete examples: medical diagnosis, spam filtering, and sensor fusion. Step-by-step inference calculations and interpretation of results.

Medical Diagnosis Spam Detection Inference Examples Real Applications

10. Python Programming for Probabilistic Reasoning

Implementing probability calculations in Python. Using NumPy for distributions, building Bayesian inference systems, and visualizing probability concepts.

NumPy Python Implementation Bayesian Code Visualization

Interactive Demonstrations

Visualize and interact with probability concepts and Bayesian inference in real-time.

Bayes' Theorem Interactive Demo

Probability Distributions Visualizer

Conditional Independence Explorer

Medical Diagnosis Simulator

Monte Carlo Simulation

Burglary-Alarm Network Demo

Practical Exercises

Work through these exercises to master probability and Bayesian reasoning.

Basic Probability Calculations

Practice computing probabilities, applying axioms, and working with sample spaces.

Bayes' Rule Applications

Apply Bayes' theorem to medical diagnosis, spam detection, and other real scenarios.

Independence & Conditional Independence

Identify independent variables and apply conditional independence to simplify models.

Bayesian Inference Problems

Complete step-by-step Bayesian inference with prior beliefs and evidence updates.

Key Concepts Summary

Probability Fundamentals:

P(A) ∈ [0, 1] for all events A
P(true) = 1, P(false) = 0
P(A ∨ B) = P(A) + P(B) - P(A ∧ B)

Bayes' Rule:

P(A|B) = P(B|A) × P(A) / P(B)
Posterior ∝ Likelihood × Prior
Update beliefs with evidence

Independence:

P(A ∧ B) = P(A) × P(B) if independent
P(A|B,C) = P(A|C) if A ⊥ B | C
Simplifies joint distributions

Key Insights:

Uncertainty is not a weakness but a feature
Probabilistic reasoning is more robust
Foundation for machine learning

SE444: Artificial Intelligence | Lecture 10: Foundations of Uncertainty & Probability