UMD Team Wins Best Paper Award at NeurIPS 2021 Workshop

Researchers from the University of Maryland recently received a best paper award for their work in developing an algorithm that helps combat adversarial attacks on machine learning models involving real-time sequential decision-making systems.

“Who is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RL,” explores how to tackle evasion attacks in reinforcement learning (RL), wherein machine learning models are trained to make a sequence of decisions. This type of adversarial attack—where data is (imperceptibly) perturbed to evade detection or to be classified as legitimate—can often be fatal for important decision-making processes.

The paper was authored by Yanchao Sun, a fourth-year doctoral student in computer science; Ruijie Zheng, a senior double majoring in computer science and mathematics; Furong Huang, an assistant professor of computer science with an appointment in the University of Maryland Institute for Advanced Computer Studies; and Yongyuan Liang, a student at Sun Yat-Sen University in China who completed an internship with Huang over the summer.

It was recognized as the best paper presented at SafeRL, a workshop that was part of the 35th Conference on Neural Information Processing Systems (NeurIPS), held virtually from December 6–14.

Widely applicable to autonomous robotics, human-machine teaming, autonomous driving systems, resource optimization, personalized intelligent healthcare systems and more, deep RL has been an increasingly important area of research in artificial intelligence and machine learning.

RL aims at finding an optimal policy/strategy for an autonomous agent— the component that makes the decision of what action to take—to interact with an environment to maximize the cumulative reward. Adversarial attacks on RL systems could take advantage of RL algorithm’s vulnerabilities and cause a catastrophic failure of the learning process.

A well-trained RL agent may be vulnerable to evasion attacks, making it risky to deploy RL models in high-stakes applications, the researchers say. For instance, an adversary may put “special stickers” on their fighter aircraft to mislead opposing forces into recognizing them as birds.

Motivated by the importance of understanding RL security and the scarcity of relevant literature, Huang’s team is focused on investigating the robustness of well-trained deep RL agents under adversarial observation perturbations. They propose a novel attacking method that searches for the optimal attack efficiently, achieving state-of-the-art attacking performance in a wide variety of environments.

Huang says their algorithm can be applied to both measure and improve the robustness of any deep RL decision-making system.

Go here to learn more about the project.

—Story by Melissa Brachfeld