PhD Proposal: Towards Human-Level Audio Intelligence: Improving Audio Understanding with Large Audio-Language Models

Talk
Sakshi
Time: 
11.02.2026 12:30 to 14:00
Location: 

Remote umd.zoom.us/my/dmanocha

Audio, comprising speech, sounds, and music, carries rich information about the physical world and is central to human communication. Large Audio-Language Models (LALMs) now perform well on recognition tasks such as speech transcription, sound classification, and captioning. However, they still struggle to reason about how sounds relate to one another, to answer expert-level questions, and to interpret long, spatial, or audio-visual input. Our research addresses these gaps by enhancing audio perception and reasoning in LALMs through better audio representations, targeted training data, and rigorous evaluation.
In this talk, I will present our work on improving the audio understanding and complex reasoning of LALMs, together with an expert-level benchmark that reveals where current models fail. I will also discuss our recent progress in long-form, spatial, and audio-visual understanding, which is key to interpreting real-world scenes. Together, these efforts move LALMs from recognizing individual sounds toward reasoning over complex auditory scenes.