When AI Goes on Defense

For a cyberattacker, one overlooked weakness can be enough. For a defender, the challenge is broader: find weaknesses and fix them before attackers exploit them.

That imbalance is becoming more consequential as artificial intelligence agents improve at writing and analyzing code. Manan Suri, a University of Maryland computer science doctoral student, is studying the other side of that equation: how AI systems can be trained to repair software vulnerabilities, not just find them.

Advised by Distinguished University Professor Dinesh Manocha, Manan’s recent work on CyberForge, developed through research with the National Institute of Standards and Technology (NIST), Université Grenoble Alpes and CNRS, focuses on one constraint facing defensive AI systems: the amount and quality of cybersecurity training data.

“AI agents are getting stronger at writing code, and as expected, I think that creates a lot of challenges for cybersecurity,” Suri said.

Uneven playing field

AI-assisted coding has increased the speed and scale at which software can be analyzed, raising the stakes for security teams trying to spot weaknesses before attackers exploit them.

“An attacker in a cybersecurity scenario only needs to find one vulnerability to get into your system, whereas a defender ideally needs to find all of them and fix all of them,” Suri said.

AI agents add another dimension to that imbalance. Unlike human operators, they can run continuously, handle multiple tasks at once and scale across large amounts of code. For Suri and his collaborators, that raised a practical research question: whether similar capabilities could be directed toward identifying and repairing weaknesses before they are exploited.

“It’s this asymmetry which creates this race, a race where we must empower defenders as much as we can,” he said.

Data ceiling

Building capable defensive systems depends in part on what they can learn from. In cybersecurity, much of that material comes from records of vulnerabilities that researchers, companies or developers have already discovered and disclosed.

Those examples are useful because they come from actual software, but they represent a finite historical record.

“At one point, you’re going to run out of the historically available data,” Suri said.

The problem becomes more pronounced when a company wants to train a model on its own software. A project may have relatively few documented vulnerabilities, leaving too little material to train an AI system around the specific structure and behavior of that codebase.

“Very often, I would imagine a scenario where an enterprise wants to train a cybersecurity agent that’s specific to their own project,” Suri said. “Individual projects don’t have enough exposed vulnerabilities.”

Synthetic data offers one way to expand that pool, but artificially created weaknesses don't necessarily reflect how vulnerabilities emerge in working software. CyberForge was developed to generate examples using information already contained within a real project.

Building CyberForge

This approach works with open-source C and C++ repositories and looks for places where code can be altered to introduce a security weakness without breaking the software’s normal behavior.

To create a useful training example, CyberForge must introduce a vulnerability without stopping the software from working normally. The researchers can then use a specific input to trigger the weakness and show the model the difference between secure and vulnerable code.

“Given limited historic data, you use the signals from the project to create realistic synthetic vulnerabilities,” Suri said. “What this allows the agent to learn is how to defend itself in case of a real vulnerability.”

The project emerged during Suri’s spring semester as a Professional Research Experience Program fellow at NIST. His background focused on AI agents, while team members brought experience in cybersecurity datasets, vulnerability analysis and standards.

The research brought together researchers from NIST, UMD, Université Grenoble Alpes, and CNRS. In addition to Suri and Manocha, the team included Amine Lbath, Aurelien Delaitre, Vadim Okun, Massih-Reza Amini and Ram D. Sriram. 

“I got to work with people who came from a background of cybersecurity,” Suri said. “This is a group of people who have released major datasets and reports and standards for vulnerabilities.”

That combination of expertise also affected how the team evaluated possible research directions. Suri said subject-matter expertise helped distinguish technically plausible ideas from approaches that would be less useful in practice.

“When you’re solving a domain-specific problem, the value that experts bring to that problem is immense,” he said.

Measuring defense

The researchers then tested whether training on CyberForge-generated vulnerabilities made the models better at protecting software. Across several experiments, performance increased by about 3.3 to 15 percentage points, Suri said.

“We wanted to see where open-source models stood on cybersecurity tasks and whether training them on this new data could improve their performance,” he said.

On one benchmark, a model successfully defended against vulnerabilities about 58% of the time before CyberForge training. After training, that figure rose to 72.7%.

The researchers also tested whether the training could help beyond the C and C++ projects CyberForge used. Some models improved on security tasks involving software written in Python, Java and JavaScript.

“We found that the training could carry over to other languages and other cybersecurity tasks,” Suri said.

Manocha said the results point to multiple benefits.

“This approach offers benefits for AI models and cybersecurity,” Manocha said. “It provides higher-quality training signals, scalable generation, broad vulnerability coverage and cross-language generalization. For cybersecurity, it can help produce more capable defense agents and provide coverage beyond known attacks.”

CyberForge addresses the training-data side of defensive AI, but Suri said broader work will also be needed on how security agents are deployed and coordinated across software systems. For defenders, the underlying challenge remains the same: they have to account for more possible weaknesses than an attacker needs to exploit.

“It’s going to be a race,” Suri said. “We should make our agents as well equipped as they can to win.”

—Story by Samuel Malede Zewdu, CS Communications