Lecture 12: Decision Theory & Rational Action Under Uncertainty

From Probability to Optimal Action Selection

Combining belief with preference to make rational choices
Back to Course Overview

Building on Lecture 11

In Lecture 11, we learned how Bayesian Networks represent uncertainty and compute beliefs. We can answer questions like: "What's the probability of disease given these symptoms?"

But knowing probabilities isn't enough for an intelligent agent. We need to act. Decision Theory combines probability with utility (preferences) to answer: "What should I do to maximize expected benefit?"

Lecture Overview

Decision theory is the foundation of rational action under uncertainty. It provides a principled framework for choosing actions when outcomes are uncertain and preferences matter.

Key Concepts: Utility Functions, Expected Utility, Decision Networks, Value of Information, MEU Principle
Applications: Medical Decisions, Robot Planning, Autonomous Vehicles, Game Playing, Resource Allocation

The Central Question

An intelligent agent faces a fundamental challenge:

How should I choose actions when I'm uncertain about outcomes?
Naive Approaches:
  • Choose action with best possible outcome (too optimistic)
  • Choose action with best worst-case outcome (too pessimistic)
  • Maximize expected outcome value (ignores preferences)
  • Random choice (not rational)
Decision Theory Solution:
  • Represent preferences with utility functions
  • Compute expected utility for each action
  • Choose action that maximizes expected utility (MEU)
  • Rational, principled, provably optimal

Main Topics

1. Motivation & Introduction

Why deterministic planning fails in uncertain worlds. Understanding rewards, expected utility, and the transition from passive inference to active decision-making.

Desert Navigation MDP Dice Game Demo Reward Visualization Expected Utility
2. MDP Fundamentals

Understanding Markov Decision Process components: States, Actions, Transitions, Rewards, and Discount factors. Interactive warehouse navigation and MDP builder demos.

States (𝑆) Actions (𝐴) Transitions 𝑇(𝑠,𝑎,𝑠′) Rewards 𝑅(𝑠,𝑎,𝑠′) Warehouse Demo
3. Transition Probabilities & Dynamics

Deep dive into transition functions, stochastic dynamics, and how actions affect state changes in uncertain environments. Markov property and state transitions.

Transition Functions Markov Property Stochastic Dynamics State Evolution
4. Policy Utility & Evaluation

Understanding utility theory and computing policy values. Bellman equations for policy evaluation, expected utility calculations, and iterative methods to assess policy quality.

Policy Utility Bellman Equations Value Functions Expected Value
5. Value Iteration & Optimal Policies

Finding optimal policies through value iteration. Dynamic programming approach to compute optimal value functions and extract optimal policies from them.

Value Iteration Optimal Policies Bellman Optimality Convergence
6. Summary & Real-World Applications

Bringing it all together: MDP framework summary, real-world applications in robotics, autonomous systems, and connections to reinforcement learning.

MDP Summary Robot Planning Autonomous Systems Reinforcement Learning

Interactive Demonstrations

Explore decision theory through interactive visualizations and calculators.

Decision Tree Builder
Utility Function Explorer
Expected Utility Calculator
Decision Network Builder
Value of Information Calculator
Medical Decision Scenario

Practical Exercises

Master decision theory through hands-on problem-solving.

Utility Theory Problems

Elicit utility functions, calculate risk premiums, analyze preferences.

Decision Tree Problems

Build decision trees, compute expected utilities, find optimal policies.

Value of Information Problems

Calculate VPI, determine when to gather information, cost-benefit analysis.

Real-World Decision Problems

Apply decision theory to medicine, robotics, business, and ethical dilemmas.

Key Concepts Summary

Decision Theory Framework:
  • States (S): possible world configurations
  • Actions (A): agent's choices
  • Outcomes (O): consequences of actions
  • Probabilities (P): uncertainty quantification
  • Utilities (U): preference representation
MEU Principle:
  • EU(a) = Σ P(s|e) × U(Result(a, s))
  • Choose action a* = argmax EU(a)
  • Rational, consistent, optimal
  • Foundation for MDPs and RL
Decision Networks:
  • Extend Bayesian Networks
  • Chance, decision, utility nodes
  • Sequential decision modeling
  • Efficient inference algorithms
Value of Information:
  • VPI(E) = EU(with E) - EU(without E)
  • Always non-negative
  • Guides information gathering
  • Trade-off: cost vs. value
SE444: Artificial Intelligence | Lecture 12: Decision Theory & Rational Action Under Uncertainty