LBNL Establishes Berkeley Institute for Performance Studies

Alan Laub, former director of DOE’s SciDAC program, once said that in the field of high performance computing, “peak performance” was defined as the speed at which “the manufacturer guarantees that you can’t compute faster than that.” Although peak performance figures make for good marketing, they don’t provide much insight into actual performance.

To rectify this, for the past eight years Berkeley Lab has been developing new tools and techniques for more accurately assessing the performance of high performance computers, especially when it comes to running real-world scientific applications.

In November, many of these projects in performance characterization, modeling and benchmarking for supercomputers were brought together to comprise the Berkeley Institute for Performance Studies. Known as BIPS, this umbrella organization will be led by Kathy Yelick and encompasses the following research activities at LBNL and UC Berkeley:

The Performance Evaluation Research Center (PERC), directed by David Bailey, is one of seven SciDAC Integrated Software Infrastructure Centers (ISICs). PERC involves approximately 25 researchers at eight centers (four labs and four universities). The goal of PERC is to develop a science for understanding performance of scientific applications on high-end computer systems, and develop engineering strategies for improving performance on these systems. The project is integrating several active efforts in the high performance computing community and is forging alliances with application scientists working on DOE Office of Science missions to ensure that the resulting techniques and tools are truly useful to end users. For detailed information about PERC, go to <http://perc.nersc.gov/main.htm>.

The Berkeley Benchmarking and Optimization Group (BeBOP) is led by Kathy Yelick and James Demmel of UC Berkeley, with substantial participation by Berkeley graduate and undergraduate students. Their research areas include:

• the interaction between application software, compilers, and hardware
• managing trade-offs among the various measures of performance, such as speed, accuracy, power, storage,
• automating the performance tuning process, starting with the computational kernels which dominate application performance in scientific computing and information retrieval
• performance modeling and evaluation of future computer architectures.

The BeBOP Web site can be found at <http://bebop.cs.berkeley.edu/>.

BeBOP works closely with the UCB LAPACK/ScaLAPACK project, which focuses on new algorithms for numerical linear algebra and new, more efficient implementations of linear algebra software.

Berkeley Lab's architecture evaluation research project, led by Leonid Oliker and Yelick, is conducted by staff from CRD and the NERSC Center Division, as well as collaborators from other institutions. They evaluate emerging architectures, such as processor-in- memory and stream processing, and develop adaptable “probes” to isolate performance- limiting features of architectures. They conducted the first in-depth analysis of state-of- the-art parallel vector architectures, running benchmark studies on the Japanese Earth Simulator System (ESS) and comparison runs on Cray's X1 system. Results on the ESS demonstrated 23 times faster performance than the IBM Power3 in a node-to-node comparison. (See the September issue of CRD Report for more information on this work.)

NERSC's benchmarking and performance optimization project is carried out by NERSC staff with expertise in performance analysis. They developed the Effective System Performance (ESP) benchmark to measure system-level efficiency and the Sustained System Performance (SSP) benchmark to measure overall system application throughput.

SSP resulted in a 30 percent increase in the Seaborg system's capability and is now used in several non-DOE procurements. This team also accelerated several SciDAC application programs running on Seaborg. Read more about ESP at <http://www.nersc.gov/projects/esp.php>.