The increasing performance of AI models in various tasks is accompanied with an increasing need to guarantee that these models' outputs conform to correctness requirements. Correctness is often interpreted as safety, but for human-facing AIs it also includes behaving normatively: that is, in a way that generally follows applicable social norms. In this talk, I focus on logic-based methods for normatively constraining RL agents' behavior. I present our work on formal modeling of norms in EAU, a probabilistic extension of deontic logic, and then for constraining the training of an RL agent with requirements in EAU. Being non-Markovian, EAU requires new methods for RL training that avoid state explosion, and being probabilistic, it requires efficient ways of incorporating probabilistic model-checking. The talk will also have a tutorial component on AI alignment (and its pitfalls) more generally, and will conclude with our current ideas on equipping AI agents with a normative module.
Houssam Abbas is an Assistant Professor of Electrical Engineering and Computer Science at Oregon State University. His research interests are in the verification and control of cyber-physical systems and formal ethical theories for autonomous agents, with particular emphasis on unpiloted ground and aerial vehicles. He received the NSF CAREER award, the Best Paper Award at the Runtime Verification conference 2023, and participated in the Frontiers of Engineering Symposium of the National Academies of Engineering. Prior to OSU, he was a postdoctoral fellow at the University of Pennsylvania, and a design automation engineer at Intel.