Parameterized Sequential Decision Making Problems: Maximum-Entropy Principle based Approach

May 06, 2022, zoom / ESB 2001

Srinivasa Salapaka

UIUC, Merchanical Science and Engineering


This talk introduces a class of optimization problems - Parameterized Sequential Decision Making (para-SDM) problems, and propose a comprehensive framework to address them. Para-SDM problems cover a vast range of network logistics and planning application areas such as facility location with path optimization (FLPO), vehicle routing, sensor network design, manufacturing process parameter optimization, last mile delivery, industrial robot-resource allocation, and data aggregation, classification, and clustering algorithms. These problems include large subclasses of problems such as Markov Decision Processes (MDPs), reinforcement learning (RL), clustering, resource allocation, scheduling, and routing problems. Conceptually these problems require simultaneously determining the shortest path and allocating resources in a network, incorporating application-specific capacity and exclusion constraints, and while respecting the dynamical evolution of the network. In this regard, they generalize Markov Decision Process (MDP) problems, which have been studied more extensively. This talk will present a combinatorial-optimization framework that address this class of problems. Many physical systems in nature pose combinatorial resource allocation problems that involve large number of configurations - on the order of Avogadro’s number, that is, about 10^23 particles. Statistical Physics provides methods and tools that make it possible to analyze such large systems. This talk will present methods and algorithms that mimic free-energy principle from statistical physics to address a class of combinatorial optimization problems. In fact, this principle is viewed as maximum entropy principle (MEP) propounded by E.T. Jaynes. The resulting framework generalized important concepts from clustering/classification literature as well as tools from machine learning – especially those related to MDPs and RL.

Speaker's Bio

Srinivasa M. Salapaka received the B.Tech. degree in Mechanical Engineering from Indian Institute of Technology in 1995, the M.S. and the Ph.D. degrees in Mechanical Engineering from the University of California at Santa Barbara, U.S.A in 1997 and 2002, respectively. During 2002-2004, he was a postdoctoral associate in the Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, USA. Since January 2004, he has been a faculty member in Mechanical science and Engineering at the University of Illinois, Urbana-Champaign. He got the NSF CAREER award in the year 2005. He is an ASME Fellow since 2015. His areas of current research interest include machine learning, combinatorial optimization, and controls for scanning-probe microscopy, X-ray microscopy, and power-grid systems.