Equilibrium in Bayesian Markov Decision Processes

March 02, 2018, Webb 1100

Ignacio Esponda

UCSB, Economics


We provide an equilibrium framework for modeling the behavior of an agent who holds a simplified view of a dynamic optimization problem. The agent faces a Markov Decision Process, where a transition probability function determines the evolution of a state variable as a function of the previous state and the agent's action. The agent is uncertain about the true transition function and has a prior over a set of possible transition functions; this set reflects the agent's (possibly simplified) view of her environment and may not contain the true function. We define an equilibrium concept and provide conditions under which it characterizes steady-state behavior when the agent updates her beliefs using Bayes' rule. Unlike the case for static environments, however, an equilibrium approach for the dynamic setting cannot be used to characterize those steady states where the agent perceives that learning is incomplete. Two key features of our approach is that it distinguishes between the agent's simplified model and the true primitives and that the agent's beliefs are determined endogenously in equilibrium.

Speaker's Bio

Ignacio Esponda is an Associate Professor and occupies the Walter J. Mead Chair of Economics at the Department of Economics, UC Santa Barbara. Professor Esponda is an economic theorist who specializes in studying learning by people with possibly misspecified models and in integrating theoretical and experimental economics. His work has been published in journals such as Econometrica, the American Economic Review and the RAND Journal of Economics.

Video URL: