PhD Proposal: Trustworthy and Efficient AI Agents

AI agents based on large language models are capable of assisting with a vast array of tasks, and have already led to a fundamental shift in the way the software and knowledge-work industry operates. In order for agents to fully live up to the promise of improving productivity, they have to be trusted to carry out tasks over long running contexts, and to stay within the user’s guidelines as the task is completed.
Unfortunately we’re not fully there yet. There have been many notable instances of agents in a commercial setting making costly mistakes during customer interaction and of code agents taking down live infrastructure for companies. And model providers need to have some way of placing guardrails around their models’ behavior to ensure they are providing their service responsibly.
This proposal focuses on two aspects of this challenge: long context efficiency and trustworthy guardrails. In the long context realm I introduce a factorization of the attention block that reduces computational complexity from quadratic to linear, and a top-k selection method for attention that makes million-token contexts possible on commodity GPUs. For guardrails, I introduce a classifier-based method that is capable of enforcing custom polices at inference time and articulating explanations about the enforcement. Further, I propose to create a framework for guardrails that are effective in the code and tool-use environments in which AI agents currently operate.