The hidden bugs in supercomputing, and how an Oregon State University computer scientist is hunting them down

As one of the most trusted simulation tools in molecular dynamics, LAMMPS has been used in more than 67,000 published studies since 1995. It’s the kind of software researchers build entire careers on. So, when Manish Motwani and his students went looking for bugs in it, they didn’t expect to find much. What surprised them was that their technique not only automatically detected 8 of 20 known heterogeneous bugs, but also uncovered 2 previously unknown bugs in LAMMPS.

“If I’m a scientist running an experiment, and I’m publishing results based on those findings, I want to know how reliable the software behind them is,” said Motwani, assistant professor of computer science. It’s a question he’s spent years trying to answer, and one that’s about to get a lot more urgent as Oregon State University prepares to open the Jen-Hsun Huang and Lori Mills Huang Collaborative Innovation Complex in 2027 with one of the nation’s most powerful academic supercomputers at its core.

The hidden cost of speed

Motwani’s research sits at the intersection of software engineering and high-performance computing (HPC) — a pairing he backed into almost by accident. During a postdoc at Georgia Tech, he began collaborating with domain scientists at national labs like Sandia and Oak Ridge and noticed a gap: researchers running massive simulations often had no formal training in software testing or verification.

“They’re able to write code, but they don’t actually know whether that code is correct, or whether the results they’re getting are correct,” he said. “And now with AI, they can generate code even faster.”

The LAMMPS bugs are a case in point. Scientific applications increasingly run across heterogeneous hardware from vendors such as NVIDIA, Intel, and AMD. Differences in hardware architectures, programming models, and implementations can sometimes cause the same scientific code to behave differently across platforms.

The problem is compounding as AI enters the picture. Scientists are increasingly using AI to translate old HPC code written for CPUs into newer programming models to take advantage of new GPU hardware. Motwani’s work published this summer found that relying on a program’s original tests to check whether an AI-translated version still works correctly can be dangerously misleading: Earlier evaluations reported up to 72% correctness, while more rigorous testing found only about 32% correctness for the corresponding AI model.

“AI is smart. It can do the translation,” Motwani said. “But that doesn’t mean the translation is doing the same thing as the original program.”

Efficiency at scale

The same rigor that catches hidden bugs, Motwani argues, can also cut the enormous energy costs of running and re-testing code on a supercomputer. A single simulation might run for an hour or more, and traditional software testing — rerunning an entire test suite every time a line of code changes — becomes wildly wasteful at that scale.

Using LAMMPS as a test case again, Motwani’s team designed an energy-aware testing technique that reduced the energy consumed by regression testing by up to 97% while maintaining high test coverage. The timing matters: as power use becomes a growing concern in the HPC community, some organizations are exploring charging researchers based on energy consumed rather than “core hours” used — a shift that could reward exactly the kind of efficient testing Motwani develops.

Work like this has already drawn national attention. A student capstone project Motwani supervised, aimed at verifying AI-translated HPC code, grew into a collaboration with Oak Ridge National Laboratory, where he gave an invited talk earlier this year. The project has since caught the interest of researchers in Japan who collaborate with Oak Ridge and want to test their own HPC translation tools using Motwani’s methods.

Future plans include building an agentic AI-based porting system that improves AI translators’ output. Motwani also plans to develop a fully integrated pipeline for testing, fault localization, and program repair of HPC scientific software, with energy efficiency as a unifying constraint throughout.

Growing OSU’s HPC community

Motwani joined Oregon State in 2023, drawn less by HPC — which barely existed at OSU at the time — than by the university’s software engineering and human-computer interaction researchers. HPC’s growth on campus since then has become “an additional benefit,” and he’s now an active member of OSU’s HiPCastor research group, collaborating with colleagues across departments, including on a proposal to help forestry researchers automate their experimental workflows.

Looking ahead to the Huang Complex, Motwani hopes the new supercomputer, to be built by NVIDIA, will do more than accelerate research.

“The supercomputer will be a boon for interdisciplinary research,” he said, pointing to oceanography, robotics, and materials science as fields likely to lean heavily on the new resource. “The question is how we make sure that what we’re building on it is fast, efficient — and actually correct.”