Independent Learning in Stochastic Games: Where Strategic Decision-Making Meets Reinforcement Learning

Reinforcement learning (RL) has recently achieved great successes in many sequential decision-making applications. Many of the forefront applications of RL involve the decision-making of multiple strategic agents, e.g., playing chess and Go games, autonomous driving, and robotics. Unfortunately, classical RL framework is inappropriate for multi-agent learning as it assumes an agent's environment is stationary and does not take into account the adaptive nature of behavior. In this talk, I focus on stochastic games for multi-agent reinforcement learning in dynamic environments, and develop independent learning dynamics for stochastic games: each agent is myopic and chooses best-response type actions to other agents' strategies independently, meaning without any coordination with her opponents. I will present our independent learning dynamics that guarantee convergence in stochastic games, including for both zero-sum and single-controller identical-interest settings. Time-permitting, I will also discuss our other results along the line of learning in stochastic games, including both the positive ones on the sample and iteration complexity of certain multi-agent RL algorithms, and negative ones on the computation complexity of general-sum stochastic games.