From Transparent to Trustworthy Speech Foundation Models

Talk
Shinji Watanabe
Carnegie Mellon University
Time: 
11.19.2026 11:00 to 12:00

Recent foundation models have dramatically advanced speech and audio technologies, but their scale has also widened the gap between large technology companies and the broader research community. In this talk, I will introduce our efforts toward building open speech foundation models, particularly the Open Whisper-style Speech Model (OWSM), and discuss the groundwork required to make such models reproducible and accessible. I will discuss our experience with data collection and cleaning, scalable training, open-source recipes, and task-specific models that support the construction of large speech foundation models. These efforts illustrate how sustained and collaborative groundwork can help democratize foundation-model research. At the same time, our experience has also shown that transparency alone is not sufficient. Issues such as hallucination, test-set contamination, and problems in data, models, or software may only become apparent after a model has been built. I will discuss how we are revisiting our own pipelines, including ongoing work on OWSM v5, with greater emphasis on provenance, permissive licensing, and the ability to identify and correct our own mistakes. Ultimately, trustworthy foundation models require not only openness and reproducibility, but also a process that allows us to continuously inspect, correct, and improve the groundwork on which they are built.