|
ECE 555 Fall 2005 Tues & Thurs 10:00-11:50 169 EL
This course introduces both the analytical and computational aspects of stochastic control and performance evaluation, with applications to engineering, computer science, economics, and management science. Topics include Markov models and stochastic stability, development of control laws by dynamic programming, complete and partial information, Kalman filtering, and an introduction to machine learning. It is intended for graduate students who have some background in control and stochastic processes. Experience with Matlab is also desirable.
Prerequisites
Some exposure to control and optimization at the level of ECE 415, and exposure to stochastic processes at the level of ECE 434 or MATH 366 (or consent of instructor.)
Grading
- 20% Homework (approximately 7 problem sets)
- 50% Two 90-minute exams
- 30% Final
Resources
The following texts are on reserve:
- P. R Kumar, Stochastic systems: Estimation, identification, and adaptive control
- Torsten Soderstrom, Discrete-time stochastic systems: estimation and control.
- Meyn & Tweedie, Markov Chains & Stochastic Stability
- Meyn, Control Techniques for Complex Networks
- Steven Shreve, Lectures on Stochastic Calculus and Finance
Outline
1. Markov Models
- Overview and examples.
- Linear and non-linear models.
- Representations of pi and value functions.
- Lyapunov Theory.
2. Optimal Control
- Controlled Markov chain models.
- Markov and stationary policies.
- Numerical techniques: Policy and value iteration, LP approaches.
- Partial information
3. Linear Theory
- Linear Gaussian systems.
- Optimal linear-quadradic and minimum-variance control.
- Partial information and the Kalman filter.
4. Adaptation and Learning.
- Stochastic approximation.
- Approximate dynamic programming.
- Introduction to adaptation and machine learning.