Imagine looking thousands of feet underground to see how a fluid moves through the Earth. Its path isn’t straight—it twists and loops in every possible direction, sometimes even doubling backward before climbing again to the surface.

For years, modeling how fluids flow through materials like soil and rock has challenged researchers, requiring advanced software and immense computing power to capture the complex journeys of fluids through tight spaces and vast underground domains. Over the past several years, Lawrence Berkeley National Laboratory (Berkeley Lab) scientists have steadily advanced the state of the art in this field.

Their latest software models now allow researchers to simulate subsurface fractures in processes with more than 400 billion degrees of freedom—a level of detail previously out of reach. This ongoing progress is opening new doors for understanding underground fluid flow, with important implications for how we extract energy, store power, and manage water in manufacturing.

“We set out to model flow and transport processes in evolving fractures, which we did, but we also ended up developing a highly scalable, general-purpose computational fluid dynamics (CFD) solver for heterogeneous systems,” said David Trebotich, a staff scientist in the Lab’s Applied Mathematics and Computational Research (AMCR) Division. “This solver allows us to reach much further into continuum-scale domains with pore-scale resolution—up to 20 times more capability than before. This was possible thanks to the DOE Exascale Computing Project, which provided access to high-performance computing (HPC) resources like the Frontier supercomputer at Oak Ridge National Laboratory, as well as support for software development of the application code.”

For more than a decade, Trebotich worked alongside researchers in Berkeley Lab’s Earth and Environmental Sciences Area, Sergi Molins and Carl Steefel, to develop a novel approach that couples pore-scale flow with reactive transport, called “Chombo-Crunch.” This technique lets scientists and engineers study how fractures and pillars form in subsurface structures, such as wellbores, and visualize fluid movement with a level of detail never before possible.

Understanding how fluids flow through various subterranean materials can help reveal paths to access energy resources, such as geothermal energy sites. This knowledge can also apply to other heterogeneous systems, including battery electrodes, where it can help identify dendrite growth failure modes—a crucial factor for safer, longer-lasting batteries. It can even help reduce the amount of water needed in paper manufacturing.

Previously, researchers and engineers relied on large-scale models, called macromodels, to understand how fluids flowed through materials. However, these models often miss the small, rapid changes that occur at the pore scale as fluids move through different materials. Chombo-Crunch overcomes this limitation by capturing those fine-scale details, resulting in more accurate and reliable simulations.

A Molecular Beginning

Although the application to energy production is now a key focus, the Chombo-Crunch software has its roots in polymer physics. Trebotich initially determined the optimal flow conditions for capturing a polymer molecule in a packed bed device by fluid mechanical forces while working on the problem more than 20 years ago. This breakthrough connected with Steefel’s work to use the finite volume, embedded boundary approach to perform direct numerical simulation of reactive transport at the pore scale, which led to the development of Chombo-Crunch.

The applications are wide-ranging. The model can help improve water desalination devices by providing clearer pictures of turbulent flow, transport, and reaction processes that lead to membrane deterioration. Chombo-Crunch improves CFD simulations by using exascale supercomputers, thereby providing more precise images of how fluids actually flow through materials.

Last year, the project team set a potential record by mapping fluid flow in a 20-meter cylinder packed with over 6 million microspheres—representing proppant (particles used to keep fractures open in subsurface formations, for example during oil and gas production)—in a long fracture. The experiment was one of the most computationally intensive simulations to date, easily surpassing previous results on CPU-based machines, which had modeled about 100 billion degrees of freedom (individual variables in the simulation), Trebotich said. In this study, the team modeled more than 400 billion degrees of freedom.

“We can now simulate a problem that is four times bigger with the same resolution as the previous capability,” Trebotich said. “And we can do even more, potentially reaching 1 trillion degrees of freedom.”

Trebotich notes that the current limitation on the degrees of freedom in a fluid flow program is not computational power, but the ability to generate the complex geometry of packed spheres required for the simulation. To overcome this challenge, he and retired lab scientist Terry Ligocki developed an algorithm for packing spheres, which initially served as geometry generation for testing the scalability of the CFD solver and now represents long fractures filled with proppant. Computational fluid dynamics simulations at these extreme scales produce vast amounts of data, which can serve as a reliable reference in the absence of experimental data. The data created by this new capability will lead to new opportunities in artificial intelligence (AI) and machine learning (ML) to perform dynamic, data-informed, multiscale, multiphysics prediction.

Trebotich and collaborators describe the multiphysics coupling framework behind Chombo-Crunch in a recent publication (Trebotich et al., 2024). Scientific applications in evolving fractured porous media are discussed in Computing in Science & Engineering (Molins et al., 2024). Trebotich’s two decades of achievement in extreme-scale CFD are summarized in the Journal of Fluids Engineering (Trebotich, 2024).