ExaStar Takes Astrophysical Research to the Exascale
June 10, 2019
Keri Troutman, email@example.com, 510-486-5071
Researching explosive astrophysical phenomena such as stellar explosions and neutron star mergers gives scientists insight into the cosmic origin of the elements and the behavior of matter. But this type of research is computationally intensive and requires the power of the largest supercomputers available. Within the next few years, exascale computing can take this research to the next level.
However, to harness the power of exascale, researchers will need new code developed specifically for exascale computing. To address this need, the ExaStar project, an application development project within the Department of Energy’s (DOE) Exascale Computing Project (ECP), brings together research teams from Lawrence Berkeley, Oak Ridge, and Argonne national laboratories and Stony Brook University to collaborate on building code for astrophysics simulations that will run on exascale computers.
Berkeley Lab’s team of ExaStar researchers is focused on simulating explosive astrophysical events to address fundamental questions in nuclear astrophysics. They’ve already tackled one of the most difficult issues in this research: simulating how radiation flows through matter.
“Determining how radiation flows through matter is key to understanding the mechanism of supernovae and predicting the detectible signatures,” said Daniel Kasen, a scientist in Berkeley Lab’s Nuclear Science Division and associate professor in the Physics and Astronomy departments at UC Berkeley. “We think that it is neutrino radiation that carries energy from the hot core of the star to the outer layers and blows everything apart; being able to simulate that process has been the biggest hurdle in solving how supernovae work.”
All simulations use approximations; one of the ExaStar team’s goals has been to develop a radiation transport solver using new mathematical algorithms that reduce the degree of approximation. “We’re at the point now where we can start running supernovae calculations that have a higher fidelity treatment of neutrinos than we’ve ever done before,” Kasen said. Toward this end, the team is leveraging supercomputing resources at Berkeley Lab’s National Energy Research Scientific Computing Center (NERSC) and the Oak Ridge Leadership Computing Facility.
The ExaStar Software Ecosystem
Codes that the ExaStar team uses -- namely, SedonaEx, Castro, and FLASH5 -- all rely on AMReX, the adaptive mesh refinement (AMR) software framework developed as part of the ECP AMR Co-Design Center, which is led out of Berkeley Lab’s Center for Computational Sciences and Engineering (CCSE). These codes form ExaStar’s software ecosystem for astrophysical modeling at the exascale. While Castro and FLASH will be used to study the stellar explosions themselves, SedonaEx will enable researchers to connect the simulation results with what observers actually see from Earth. As Don Willcox, a CCSE postdoctoral computational scientist, points out, “You can't directly observe what happens inside a supernova explosion in real time. So to figure out how these supernovae work, you have to put together calculations from different physics algorithms over a very wide range of spatial and time scales. And at the end you compare your model predictions to observations to see how well they match.”
The ECP is funding multiple groups to work on building codes for exascale computers, which will be available within the next few years and promise computing power that is 50 times faster than the most powerful supercomputers available at the start of the project.
“If our simulations can effectively use that big bump in computer power, we’ll be able to go from running one or two exploratory models of supernovae to running parameter surveys of many models and actually doing predictive science,” said Kasen. “It will be a game changer.”
The explosive astrophysical events that the Berkeley Lab ExaStar team are looking at now provide a way to probe basic events in physics. “These are cosmic laboratories that reach conditions that we could never reach on Earth, so they give us insight into how nature works in these environments of extreme density and gravity,” he added.
These extreme environments can help scientists address questions of origin—how the matter that makes up the Earth and humans first came into being in the universe. “We know that most of the heavier elements in the universe came from these exploding/colliding stars,” said Kasen. “We still don’t fully understand how those nuclear processes work, so we are trying to simulate the creation of the elements inside supernovae and related phenomena and finally answer that question of where the elements came from.”
In addition, while scientists have known for decades that supernovae exist, they do not yet understand how they work in detail. “Getting exascale machines will give us the raw power, but to make use of it we have to do a lot of work to get our codes effectively running on these new architectures,” said Kasen.
About Computing Sciences at Berkeley Lab
The Computing Sciences Area at Lawrence Berkeley National Laboratory(Berkeley Lab) provides the computing and networking resources and expertise critical to advancing Department of Energy Office of Science (DOE-SC) research missions: developing new energy sources, improving energy efficiency, developing new materials, and increasing our understanding of ourselves, our world, and our universe. ESnet, the Energy Sciences Network, provides the high-bandwidth, reliable connections that link scientists at 40 DOE research sites to each other and to experimental facilities and supercomputing centers around the country. The National Energy Research Scientific Computing Center (NERSC) powers the discoveries of 7,000-plus scientists at national laboratories and universities. NERSC and ESnet are both Department of Energy Office of Science National User Facilities. The Computational Research Division (CRD) conducts research and development in mathematical modeling and simulation, algorithm design, data storage, management and analysis, computer system architecture and high-performance software implementation.
Berkeley Lab addresses the world's most urgent scientific challenges by advancing sustainable energy, protecting human health, creating new materials, and revealing the origin and fate of the universe. Founded in 1931, Berkeley Lab's scientific expertise has been recognized with 13 Nobel prizes. The University of California manages Berkeley Lab for the DOE’s Office of Science. The DOE Office of Science is the United States' single largest supporter of basic research in the physical sciences and is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.