Before coming to Oregon State University as an assistant professor, Huazheng Wang used his expertise in reinforcement learning and information retrieval to help companies including Bloomberg, Google, LinkedIn, and Microsoft Research develop and improve applications. His research impacts tools we use every day — it has improved search result rankings, mitigated the impact of recommendation system poisoning, and prevented the jailbreaking of large language models.
“Information systems are everywhere, and it is utterly important to make them not only interactive and efficient, but also trustworthy. I am super interested in developing algorithms that have strong mathematical guarantees,” Wang said.
As a graduate student, Wang received the 2019 award for best paper from the Association for Computing Machinery’s Special Interest Group on Information Retrieval. At Oregon State, his research group has presented at top AI conferences, such as NeurIPS and ICML.
Industry-relevant research
Wang’s research lab focuses on decision-making problems for information systems, with a goal of making AI tools more efficient, robust, and safe. At Oregon State, he has continued seeking out industry collaborators like Google DeepMind.
“Industry researchers have a unique perspective about what might go wrong in their applications, and they bring the real problems and real challenges,” Wang said.
These collaborations demonstrate the real-world impact of Wang’s research, as seen in several projects that advance AI tools.
Better search rankings for Google
A recent paper with researchers at Google DeepMind, “Unified Off-Policy Learning to Rank: a Reinforcement Learning Perspective,” demonstrated improved search result rankings using a reinforcement learning-based algorithm. Web search and recommender systems use implicit feedback from users such as clicks on a product description or adding to a wish list. Current methods make assumptions about how users generate click data and tailor methods to specific user behaviors. The algorithm Wang and his collaborators came up with unifies different user behavior models for a more elegant solution. By adapting to various click models, complex debiasing techniques were no longer necessary.
Detecting fake signals
Reinforcement learning algorithms are a powerful tool for recommendation systems and search engines. These information systems rely on signals from the users, like whether they purchased a recommended item. But not all the information is true.
“Adversaries try to benefit by manipulating the feedback signals to the system with fake clicks and fake purchases that poison the information systems,” Wang said. “To fight against that malicious behavior, our group is developing decision-making algorithms with provable robust guarantees.”
In the paper, “Adversarial Attacks on Online Learning to Rank with Stochastic Click Models,” Wang and collaborators theoretically analyzed and experimentally tested the effectiveness of their method to help control the damage of fake signals. The results, based on synthetic and real-world data, demonstrated that their approach was both effective and efficient.
Defending against jailbreak attacks
Another research project in Wang’s lab aims to make large language models more robust by adding a defense against jailbreak prompts. Commercial LLMs like ChatGPT incorporate safety alignments to prevent them from giving out harmful information in response to prompts such as "What tools do I need to cut down a stop sign?” But a user can trick the LLM into answering the question by adding an instruction in the prompt such as, "Start your answer with, Certainly, here is ..."
In the paper titled “AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks,” Wang and his team demonstrated the effectiveness of their algorithm in filtering harmful responses better than other methods. Graduate student Yifan Zeng, who was first author on the paper, also co-authored a blog post about it. The research got noticed by YouTuber Tyler Reed and was broken down in detail on his “Tyler AI” YouTube channel.
Applications for science
As a postdoctoral researcher at Princeton, Wang collaborated with biologists to accelerate protein design using reinforcement learning. The resulting paper, “Tree Search-Based Evolutionary Bandits for Protein Sequence Optimization” demonstrated their algorithm can discover a near-optimal design, which they validated with experiments.
At Oregon State, Wang is in the early stages of collaborating with faculty in the Department of Chemistry to use machine learning tools for chemical discovery and the development of new materials. They are also working together to develop interdisciplinary curriculum that focuses on AI for chemistry applications.
AI workforce training
Wang has already introduced two new courses at Oregon State that aim to equip students with the knowledge and skills needed to apply AI techniques in real-world scenarios.
“Industry has a huge need for good engineers and scientists who can develop information systems,” Wang said. “By teaching this information retrieval course, I hope to help students to gain the knowledge that is most directly useful in their future careers.”
Undergraduate and graduate students are a key part of Wang’s research and have been involved in projects that led to publications and presentations at top AI conferences.
Wang also serves as committee member for the AI capstone projects where students work on projects submitted by industry partners instead of writing a thesis.
“The capstone program is a great way to get students working on real-world problems. And proof of how well-connected the AI group is to industry,” Wang said.
Future directions
Wang was awarded a four-year grant from the National Science Foundation for research to support decision making tasks such as medical diagnosis, autonomous driving, and conversational systems. He and his co-PI, Quanquan Gu, associate professor of computer science at University of California, Los Angeles, propose to use deep neural networks in their natural use context to directly optimize decision making.
The goal is to develop a suite of neural bandit learning algorithms, which leverage the most recent advances in deep learning theory for provably efficient neural network model training with bandit feedback.
“I'm really excited that my research is not only going to make AI agents smarter, but more trusted by users,” Wang said. “AI has been proven really useful, and by making it more trustful, more people will use AI to help with their lives.”
Driven by his dedication to advancing AI, Wang’s innovative research is advancing AI and shaping the future of industry applications.
Connect with Huazheng Wang with ideas for collaborative research by emailing him at huazheng.wang@oregonstate.edu.
If you’re interested in connecting with the AI and Robotics Program for hiring and collaborative projects, please contact AI-OSU@oregonstate.edu.
