AI-MI Seminar Series: Sharon Li (University of Wisconsin-Madison)

Reliability Along the Way: Progress and Failure Signals in AI Agents

AI systems are rapidly evolving from models that generate responses to agents that execute long-horizon workflows, interact with tools and environments, and take increasingly consequential actions. This shift changes the nature of the reliability problem. Knowing whether an agent ultimately succeeds is no longer enough: we need to understand whether it is making progress along the way, where it fails, and how such signals can be used to improve future behavior.

In this talk, I will present our recent work toward reliability mechanisms that operate along an agent’s trajectory. I will first discuss how uncertainty changes in interactive agentic systems and introduce progress advantage, a per-step signal that can be recovered directly from standard RL post-training without additional annotations or reward-model training. I will then turn to failure diagnosis: rather than relying on expensive LLM judges or manually labeled failures, we explore whether an agent can learn from successful executions alone to recognize when a new trajectory begins to go off track. Finally, I will show how intermediate progress signals can provide more informative credit during reinforcement learning, helping long-horizon agents learn which actions actually contributed to success or failure.

Sharon Li, Associate Professor, Department of Computer Sciences, College of Computing and AI, University of Wisconsin-Madison

I am an Associate Professor in the Department of Computer Sciences and the College of Computing and Artificial Intelligence at the University of Wisconsin-Madison. I am a member of machine learning@uw-madison and a faculty affiliate with the Data Science Institute. Previously, I was a postdoc researcher in the Computer Science department at Stanford University, where I worked with Christopher Ré. I received my PhD from Cornell University in 2017, where I was advised by John E. Hopcroft and worked closely with Kilian Q. Weinberger.

My research focuses on the foundations of reliable intelligence. We develop learning systems that can recognize the limits of their competence, reason under uncertainty, and make reliable decisions in the open world. Currently, we focus on reliable LLM and agentic systems, including diagnosing and improving long-horizon agent behavior, reasoning reliability, and hallucination detection. We also develop new post-training optimization and reinforcement learning approaches to shape model behavior toward designed outcomes. Ultimately, our goal is to develop AI systems whose behaviors can be rigorously understood and predicted, enabling principled control throughout the model lifecycle, from training to deployment.

Thursday, November 12, 2026, 10:00 – 11:00 am ET · Watch live on YouTube @AIMaterialsInstitute