Primal-Dual Algorithms for Learning in Constrained Markov Decision Processes

Abstract

Many real-world tasks in fields such as robotics and control can be formulated as constrained Markov decision processes (CMDPs). In CMDPs, the objective is usually to optimize the return while ensuring some constraints being satisfied at the same time. The primal-dual approach is a common technique of addressing CMDPs. It rewrites the original optimization problem of CMDPs into its equivalent Lagrangian form. In this thesis, we deliver an overview of CMDPs and the primal-dual approach, explain several algorithm designs adopting the primal-dual approach under different learning settings in terms of simulator types, and provide analysis of these algorithms.