Lecture 14: Probabilistic Reasoning over Time

Lecture 14: Probabilistic Reasoning over Time¶

AIMA Chapter 14 — 1 hour¶

Learning Objectives¶

• Model temporal processes with transition and sensor models

• Perform filtering, prediction, and smoothing

• Use hidden Markov models and Kalman filters

• Understand dynamic Bayesian networks

Time and Uncertainty¶

• States: X₀, X₁, X₂, ...

• Observations: E₁, E₂, ...

• Markov assumption: Current state depends only on previous

Transition and Sensor Models¶

• P(Xₜ|Xₜ₋₁): Transition model

• P(Eₜ|Xₜ): Sensor model

• Stationary: Same for all t

Inference Tasks¶

• Filtering: P(Xₜ|e₁:t) — current state

• Prediction: P(Xₜ₊ₖ|e₁:t) — future

• Smoothing: P(Xₖ|e₁:t) for k < t — past

Filtering (Forward)¶

• Recursive: P(Xₜ|e₁:t) = α P(eₜ|Xₜ) Σₓₜ₋₁ P(Xₜ|Xₜ₋₁) P(Xₜ₋₁|e₁:t₋₁)

• Time: O(|X|²) per step

• Space: O(|X|)

Smoothing¶

• Forward-backward: P(Xₖ|e₁:t) ∝ P(Xₖ|e₁:k) P(eₖ₊₁:t|Xₖ)

• Backward message: P(eₖ₊₁:t|Xₖ)

Most Likely Sequence¶

• Viterbi: Dynamic programming

• δₜ(x): Probability of most likely path to Xₜ=x

• Backpointers: Reconstruct path

Hidden Markov Models¶

• States: Discrete

• Observations: Discrete or continuous

• Matrix form: Transition A, emission B

HMM: Localization¶

• States: Grid cells

• Actions: Move (noisy)

• Observations: Sensors (noisy)

• Filtering: Update belief with move and sense

Kalman Filters¶

• States: Continuous (Gaussian)

• Linear: Transition and observation

• Update: Closed-form, O(n³)

• Extended KF: Linearize nonlinear

Dynamic Bayesian Networks¶

• DBN: BN over time

• 2-slice: P(Xₜ|Xₜ₋₁), P(Eₜ|Xₜ)

• Inference: Exact (junction tree) or approximate

Summary¶

• Filtering, prediction, smoothing: Forward, forward-backward

• HMM: Discrete states, matrix form

• Kalman: Continuous, linear, Gaussian

• DBN: General temporal BN

References¶

• Russell & Norvig, AIMA 4e, Ch. 14

• Chapter PDF: chapters/chapter-14.pdf

Questions?¶

Next lecture: Probabilistic Programming (Chapter 15)