For stochastic control problems with uncountable state and action spaces, the computation of optimal policies is known to be prohibitively hard. In this talk, we will present conditions under which finite models obtained through quantization of the state and action sets can be used to construct approximately optimal policies. Under further conditions, we obtain explicit rates of convergence to the optimal cost of the original problem as the quantization rate increases. We study various conditions on the controlled system models such as weak continuity, strong continuity and continuity in total variation, and the state and action spaces. Using information theoretic arguments, we show that the convergence rates are order-optimal for a large class of problems.
We then extend our analysis to decentralized stochastic control problems, also known as team problems, which are increasingly important in the context of networked control systems. We present new existence results for optimal policies, and building on these we show that for a large class of sequential dynamic team problems one can construct a sequence of finite models obtained through the quantization of measurement and action spaces whose solutions constructively converge to the optimal cost. The celebrated counterexample of Witsenhausen is an important special case that will be discussed in detail. (Joint work with Naci Saldi and Tamas Linder).
Serdar Yuksel received his B.Sc. degree in Electrical and Electronics Engineering from Bilkent University in 2001; M.S. and Ph.D. degrees in Electrical and Computer Engineering from the University of Illinois at Urbana-Champaign in 2003 and 2006. He was a post-doctoral researcher at Yale University before joining Queen’s University as an Assistant Professor of Mathematics and Engineering in the Department of Mathematics and Statistics, where he is now an Associate Professor. He has held visiting positions at KTH Royal Institute of Technology, Sweden and Bilkent University, Turkey; and has been awarded the 2013 CAIMS/PIMS Early Career Award in Applied Mathematics. He is an associate editor for the IEEE Transactions on Automatic Control. His research interests are on control theory, information theory and probability.
