PhD Proposal: PhD Preliminary Generative Models for Visual Content Creation: Video, Storytelling, and Documents
Generative models now produce images and videos of high visual quality, yet the people who make content with them still struggle to control what comes out.
This proposal presents my research on that control problem across three media. In video, a story told through several prompts must stay coherent across scene boundaries. In storytelling, characters must speak in a way that matches what is on screen and what they said before. In documents, a new background must not damage the text, layout, and readability of a page. The completed work addresses each medium with a training-free method built on pretrained generative models. Text2Story keeps multi-scene video coherent, Action2Dialogue gives characters grounded dialogue and speech, and BGGEN generates document backgrounds that preserve the text, with an accepted follow-up, Dynamics-Inspired Diffusion, that removes the explicit masks.
The proposed research continues this line in two thrusts. The first brings three manuscripts now under review (NEGATE, DCR, and TPD), which control what pretrained video models get wrong by default, together into one control layer for general video generation, in the same direction as the planned industry internship. The second, part of an interdisciplinary research collaboration, turns news articles into short explainer videos for viewers with no prior knowledge of the events, with the goal that viewers understand the article from the video as well as readers do from the text. The goal for the dissertation defense is Spring 2028 or earlier.