ECP Pagoda Project Rolls Out First Software Libraries
November 1, 2017
Just one year after the U.S. Department of Energy's (DOE) Exascale Computing Program (ECP) began funding projects to prepare scientific applications for exascale supercomputers, the Pagoda Project—a three-year ECP software development program based at Lawrence Berkeley National Laboratory—has successfully reached a major milestone: making its open source software libraries publicly available as of September 30, 2017.
Led by Scott B. Baden, Group Lead of the Computer Languages and Systems Software (CLaSS) Group within Berkeley Lab’s Computational Research Division, the Pagoda Project’s libraries are designed to support lightweight global address space communication for exascale applications. The libraries take advantage of the Partitioned Global Address Space (PGAS) model to emulate large, distributed shared memories. By employing this model, which allows researchers to treat the physically separate memories of each supercomputer node as one address space, the Pagoda libraries will be able to leverage available global address hardware support to significantly reduce the communication costs of moving data—often a performance bottleneck in large-scale scientific applications, Baden explained.
“Our job is to ensure that the exascale applications reach key performance parameters defined by the DOE,” he added.
Thus this first release of the software is as functionally complete as possible, Baden emphasized, covering a good deal of the specification released last June. “We need to quickly determine if our users, in particular our ECP application developer partners, are satisfied,” he said. “If they can give us early feedback, we can avoid surprises later on.”
GASNet-EX and UPC++
The Pagoda software stack comprises a communication substrate layer, GASNet-Ex, and a productivity layer, UPC++. GASNet-Ex is a communication interface that provides language-independent, low-level networking for PGAS languages such as UPC and Coarray Fortran, the UPC++ library and for the Legion Programming Language. UPC++ is a C++ interface for application programmers that creates “friendlier” PGAS abstractions above GASNet-Ex’s communication services.
“GASNet-Ex, which has been around for over 15 years and is being enhanced to make it more versatile and performant in the exascale environment, is a library intended for developers of tools that are in turn used to develop applications,” Baden explained. “It operates at the network hardware level, which is more challenging to program than at the productivity layer.” The GASNet-Ex effort is led by Pagoda co-PI Paul Hargrove and was originally designed by Dan Bonachea, who jointly develops the software. Both are members of CLaSS.
As the productivity layer, UPC++ sits at a slightly higher level over GASNet-Ex, in a form appropriate for applications programmers. The goal of this layer is to impose minimal overheads in exchange for hiding considerable idiosyncratic detail, so users are satisfied with the benefits obtained by increased productivity.
Over the past year, the Pagoda team worked closely with several Berkeley Lab partners to develop applications and application frameworks, including the Adaptive Mesh Refinement Co-Design Center (AMReX), Sparse Solvers (ECP AD project) and ExaBiome (ECP AD Project). They also worked with several industry partners, including IBM, NVIDIA, HPE and Cray, and over the next few months will be meeting with all of the major vendors who are vying to build the first exascale computer or the components that will go into those computers.
“We are part of a large community of ECP developers,” Baden said. “And the ECP wants to deploy a software stack, a full set of tools, as an integrated package that will enable them to ensure that the pieces are compatible, that they will all work together. I am fortunate to be working with such a talented team that is highly motivated to deliver a vital component of the ECP software stack.” This team includes other members of CLaSS—Steve Hofmeyr and Amir Kamil (at the University of Michigan)—as well John Bachan, Brian van Straalen and Mathias Jacquelin. Bryce Lelbach, now with NVIDIA, also made early contributions.
Now that they are publicly available, the Pagoda libraries are expected to be used by other ECP efforts and supercomputer users in general to meet the challenges posed not only by the first-generation exascale computers but by today’s petascale systems as well.
“Much of the ECP software and programming technology can be leveraged across multiple applications, both within ECP and beyond,” said Kathy Yelick, Associate Lab Director for Computing Sciences at Berkeley Lab, in a recent interview with HPCwire. For example, AMReX, which was launched last November and recently announced its own first milestone, released its new framework to support the development of block-structured AMR algorithms, and at least five of the ECP application projects are using AMR to efficiently simulate fine-resolution features, Yelick noted.
For the remaining two years of the Pagoda project, the team will be focused on application integration and performance enhancements that adeptly leverage low-level hardware support, Baden noted.
About Computing Sciences at Berkeley Lab
High performance computing plays a critical role in scientific discovery. Researchers increasingly rely on advances in computer science, mathematics, computational science, data science, and large-scale computing and networking to increase our understanding of ourselves, our planet, and our universe. Berkeley Lab’s Computing Sciences Area researches, develops, and deploys new foundations, tools, and technologies to meet these needs and to advance research across a broad range of scientific disciplines.
Founded in 1931 on the belief that the biggest scientific challenges are best addressed by teams, Lawrence Berkeley National Laboratory and its scientists have been recognized with 16 Nobel Prizes. Today, Berkeley Lab researchers develop sustainable energy and environmental solutions, create useful new materials, advance the frontiers of computing, and probe the mysteries of life, matter, and the universe. Scientists from around the world rely on the Lab’s facilities for their own discovery science. Berkeley Lab is a multiprogram national laboratory, managed by the University of California for the U.S. Department of Energy’s Office of Science.
DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit energy.gov/science.