UMD Researchers Receive EMNLP Best Paper Award

University of Maryland Department of Computer Science Professor Jordan Boyd-Graber and Associate Professor Rachel Rudinger, along with doctoral student Neha Srikanth, received a Best Paper Award at the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP) for research examining how humans and artificial intelligence systems respond to questions.

Their paper, titled "DiscoTrace: Representing and Comparing Answering Strategies of Humans and LLMs in Information-Seeking Question Answering," was one of two papers selected for the award. The conference was held Oct. 24-29 in Budapest, Hungary. It focuses on natural language processing, the study of how computers interpret and generate human language.

Boyd-Graber and Rudinger co-advised Srikanth on the project, which examined why AI-generated answers can sound similar even when users ask different kinds of questions.

"AI answers are like informational junk food," Boyd-Graber said. "They sound good, but don't do what they need to do."

The researchers looked beyond whether an answer contains correct facts to examine how respondents approach a question. Some answers address one interpretation, while others cover several possibilities, challenge an assumption or ask for clarification.

Rudinger said the project grew out of earlier research on vague questions or those based on questionable assumptions. That work explored whether AI systems could identify the different meanings a question might have. The findings led to another question about how those systems should respond.

"But once you've identified those different interpretations," Rudinger said, "there is still the issue of: what strategy should you actually use to respond to the question?"

She offered the question "Are chiropractors real doctors?" as an example. Someone might be asking about licensing requirements, scientific evidence supporting chiropractic treatment or how other medical professionals view chiropractors. A respondent could address one meaning, cover several or ask the person to clarify.

To compare these approaches, the team developed DiscoTrace, a framework that divides answers into smaller sections and identifies each section's purpose. Drawing on linguistic theory, it tracks how a response supplies information, explains context or addresses the assumptions behind a question.

The researchers analyzed 18,968 human-written responses to 2,700 questions from nine Reddit communities, then compared those patterns with answers produced by large language models, the technology used in many chatbots.

"Our hypothesis was that AI systems were relying on a more limited repertoire of answering strategies compared to humans," Rudinger said.

The analysis found that human responses varied across online communities. In specialized forums, respondents more often provided background information, sources or clarification. In broader discussion spaces, they more frequently offered direct explanations or advice.

Even communities discussing similar topics approached questions differently. In parenting forums, for instance, some respondents emphasized research-based information while others drew more heavily on personal experience.

AI-generated answers followed more consistent patterns across topics. The models also tended to address more possible interpretations of questions than human respondents, who often focused on fewer meanings.

Boyd-Graber said the work was also connected to National Institutes of Health-supported research involving Rosie, a question-answering system designed to provide information to new and expectant mothers. His involvement drew on a broader interest in what makes an answer useful to the person asking.

"The three of us spent well over a year iterating on the ideas and methods together," Rudinger said, "and Neha especially put an incredible amount of effort and care into the project as lead student author."

The researchers next plan to investigate ways to guide AI systems toward particular answering strategies, building on the categories identified through DiscoTrace.

"Ultimately we want AI systems to be able to provide users with helpful answers and adapt to their individual needs," Rudinger said, "which may require invoking different answering strategies at the appropriate times."

—Story by Samuel Malede Zewdu, CS Communications