Clearing a path through the AI wilderness

Most deep learning models, especially in scientific domains, are notoriously difficult to interpret. Landscaper creates visual snapshots of what’s known as a loss landscape — a vast, complex space that shows how well a machine learning model performs under different conditions. These visuals act like maps that help researchers explore how a model behaves as it learns and changes.

In these visualizations, the landscape may resemble a mountain range, where lower valleys often represent better-performing models and high peaks can signal trouble spots. Scientists use these maps to see how an AI system might “travel” through this terrain during training, whether it’s heading toward a good solution or getting stuck in a less helpful one.

“Landscaper introduces powerful new tools that help scientists and engineers peek under the hood of these AI models and understand what’s really going on inside,” Maciejewski says. “This kind of visibility is essential for trust in high-stakes applications like drug discovery and industrial chemistry.”

For Jiaqing Chen, a computer science doctoral student and core contributor to the project since 2022, the collaboration has been a deeply rewarding experience.

“The goal of our work is to create a bridge between scientific machine learning and visualization,” Chen says. “By combining topology, statistics and visual design, we’ve created tools that make some of the most complex models in science easier to understand and improve.”

Chen’s leadership on the project has included multiple published papers, hands-on research visits to Berkeley Lab and the development of Landscaper itself.

“My onsite experience at Berkeley Lab has been truly rewarding,” he says. “Every summer, I’ve had the opportunity to visit and work with researchers face-to-face. These in-person sessions have been essential to driving the project forward and refining its direction.”

Smart data beats more data

To prove its worth, the researchers put Landscaper to the test on a particularly gnarly real-world problem: predicting the outcome of a chemical reaction called olefin hydroformylation.

The reaction is a common large-scale industrial process used in the production of many everyday products, such as detergents, plastics and alcohols. But results are hard to predict due to a lack of data about the complex molecular behavior.

Scientists have developed 3D graph neural networks, a type of AI that takes the 3D shape of molecules into account to make better predictions. These models showed promising performance, but no one really understood how or why they worked.

That’s where Landscaper came in. Using Landscaper, the ASU-led team mapped out the models’ loss landscapes, visually rendering distinct valleys, tricky saddle points and varying levels of “sharpness” in the terrain. One of the team’s most exciting discoveries is that using expert knowledge about chemistry to create more training examples improved AI performance more than giving the system more information about each molecule.

As part of the test, the team generated different 3D shapes of the same molecule and showed how it might wiggle into different shapes. By showing the model many variations, it learned to draw better conclusions than when the team added more information or more features. Simply piling on extra information didn’t guarantee better results.

Rostyslav Hnatyshyn, a Fulton Schools computer science doctoral student on the team, joined the project to make Landscaper easier for others to use. He optimized performance and built a model-agnostic interface.

“Machine learning models are often deployed without being properly evaluated,” he says. “Our visualizations help developers understand how well their models are trained and whether they might be overfit or underperforming in subtle ways.”

Landscaper is open source, free and designed to be easy to use. This makes it practical for scientists who might not have deep backgrounds in AI but need to evaluate and troubleshoot model performance.

Maciejewski says that ensuring the tool would be useful in the real world was an important aspect of the project, and more generally, of his other work in progress in the School of Computing and Augmented Intelligence.

“An essential part of our research mission is making sure that AI is not just accurate, but also explainable, reliable and usable,” he says. “And we’re focused on the places where it can make the biggest impact.”

But perhaps the best part of the project is the richly rewarding student experiences. Hnatyshyn says he’s grateful for his time at Berkeley Lab over the summer.

“The collaborative atmosphere is inspiring,” he says. “Every conversation pushed me to think more deeply about the research and reminded me how much there still is to learn.”