Introduction
In the controlled, predictable environments of a laboratory or a warehouse, a robotic arm can stack boxes with surgical precision, and a robot vacuum can navigate a room without a single misstep. But transport those same systems into the chaotic, unscripted reality of the real world — an open environment that defies perfect mathematical modeling — and their reliable behavior can quickly unravel.
Addressing this fundamental gap is the core mission of Sandhya Saisubramanian, assistant professor of computer science at Oregon State University and director of the Intelligent Reliable Autonomous Systems (IRAS) Lab. Her research focuses on developing algorithms that help autonomous agents recognize their own operational limits, pinpoint the underlying causes of unexpected failures, seek information to improve decision-making, and quickly modify their actions. This line of inquiry has earned her a prestigious National Science Foundation CAREER award. Her five-year project aims to pioneer a novel paradigm in artificial intelligence: introspective autonomy.
While much of the existing AI literature focuses on fixing a robot’s behavior after a failure, Saisubramanian is designing frameworks that enable autonomous agents to be introspective and proactive. Rather than assuming their models are correct, introspective systems developed in this project will actively evaluate the adequacy of their own knowledge and determine what information is needed to improve future decisions. By enabling robots to reason about the limitations and uncertainty in their internal models, she is training them to safely navigate unfamiliar situations without constant human intervention.
Identifying gaps in agent knowledge
When an AI system operates in the real world, it often does so based on incomplete information. A designer might accidentally overlook a specific environmental variable, or cultural and regional differences might introduce unexpected habits that were not accounted for during the development phase. These missing pieces give rise to undesired disruptions that occur alongside a system’s primary objective.
A central challenge is that autonomous agents often do not know when their internal understanding of the world is incomplete. A robot may fail because its model of the environment is inaccurate, because it is missing important information, or because the world has changed in an unexpected way.
Consider a standard robot vacuum. It is programmed to clean a floor, but it might lack the common sense or background knowledge to realize that spraying water near a power socket or dragging spilled food across a rug is entirely unacceptable. Saisubramanian's research investigates how autonomous systems can assess the quality of their own models and determine what information would be most useful for improving future decisions.
Credit: Karl Maasdam
Moving beyond introspection
Once an autonomous system recognizes limitations in its knowledge, it must still decide how to act. One component of Saisubramanian’s CAREER project is to develop efficient planning methods that explicitly reason about multiple objectives and side effects. Rather than forcing a robot to learn everything from scratch, her framework uses a lexicographic ordering of objectives. The framework enables autonomous agents to balance multiple objectives, whose relative importance may change across different operating contexts. For example, in some contexts, the primary goal is to complete the assigned task, and the secondary goal is to minimize negative side effects. By introducing a “slack” parameter — an allowable deviation from absolute peak performance — the robot is given the flexibility to prioritize safety. For instance, a vacuum cleaner might choose to leave a strip of the floor uncleaned if approaching it would risk splashing water onto a nearby wall.
To learn these boundaries, Saisubramanian’s algorithms enable autonomous agents to decide when, where, and how to seek additional information that would be most useful. In addition to learning from human feedback, the project will also utilize environment-shaping methods in which humans and robots operate as a team. Instead of just correcting the robot’s code, a human might place a protective sheet over a delicate rug. The robot’s planning module then senses this change and adaptively recalculates its trajectory, maintaining efficiency without triggering adverse side effects.
Crucially, Saisubramanian aims to gather these learning course corrections subtly. “I want these systems to be capable of learning from both direct and indirect signals that are available to it in the environment without requiring frequent user intervention,” she said.
Credit: Karl Maasdam
From simulation to campus streets
Saisubramanian’s work is not confined to theoretical models. Her lab will upload these advanced decision-making frameworks onto off-the-shelf hardware, specifically a Jackal mobile robot from Clearpath and a Kinova tabletop manipulator. On the sidewalks and streets of OSU, the navigation-focused Jackal will face a semi-structured open world filled with unpredictable pedestrians, cyclists, and temporary construction closures. Indoors, the Kinova robotic arm will test its contextual planning skills on manipulation tasks like rearranging items, stacking blocks, and wiping down kitchen countertops.
A major challenge is determining which information sources are reliable and informative to accelerate robot learning. When learning from human feedback, the robot must learn how to filter out human biases and noise. Humans have habits that robots simply do not share, and a robot must be able to discern true environmental signals from idiosyncratic human behavior.
“A human, for example, may need a coffee in the morning to function better,” Saisubramanian said. “But I don’t want the robot to think that every time it starts working on a task, it should start with a cup of coffee.”
Furthermore, a robot must learn that some real-world changes, like a road closure, are temporary disruptions rather than permanent alterations to the map.
Cultivating trust
The long-term goal is to move beyond autonomous systems that merely execute instructions and toward systems that can adapt through introspection. By formalizing the principle of introspection, Saisubramanian’s work aims to lay the foundation for future autonomous systems that are human-aligned from the ground up, verifying their safety parameters at the exact moment of learning.
Ultimately, the objective of her lab is to shift society's view of AI mistakes and cultivate a stronger foundation for human-machine interaction.
“When we observe an autonomous agent making a mistake, it would be great if humans could have full confidence in saying ‘Oh, it made a mistake, but it can recognize why it happened and learn to overcome it’” Saisubramanian said. “That level of trust would be an ideal long-term goal of this work.”
