A-Z Index | Directory | Careers

Berkeley Lab Researchers Prepare U.S. Climate Community for 100-Gigabit Data Transfers

October 30, 2009

Contacts: Linda Vu, [email protected], 510-495-2402


This 3D plot of the vorticity isosurfaces was developed using the VisIt visualization tool, for the Global Cloud Resolving Model research group. This work was performed in collaboration with Prabhat at NERSC.

As researchers around the world tackle the issue of global climate change, they are both generating and sharing increasingly large amounts of data. This increased collaboration helps climate scientists better understand what is happening and evaluate the effectiveness of possible mitigations.

But sharing these increasingly large datasets requires reliable high-bandwidth networks. To help ensure that the climate research community has the resources necessary to access, transfer and analyze the data, the U.S. Department of Energy has funded several related projects at Lawrence Berkeley National Laboratory.

The newest project, called Climate 100, will help the research community effectively use the planned 100-gigabit-per-second networks. Climate 100, funded with $201,000 under the American Recovery and Reinvestment Act, will bring together middleware and network researchers to develop the needed tools and techniques for moving unprecedented amounts of data.

"Climate 100 is a system that will integrate massive climate datasets, emerging 100 Gbps networks, and state-of-the-art data transport and management technologies to enable realistic at-scale experimentation with climate data management transport and analysis in a 100 Gbps, 100 petabyte world," says Alex Sim, a co-principal investigator of the Climate 100 project. Sim is a member of the Scientific Data Management Group at Berkeley Lab.

As Climate 100 tools are being developed, DOE's Energy Sciences Network (ESnet) will be building a prototype 100 Gbps network, the fastest network to date linking DOE supercomputing centers in California, Illinois and Tennessee. Called the Advanced Networking Initiative, the project will advance the development of 100 Gbps equipment and services to handle the ever increasing flow of critical data between research institutions. The initiative, managed by ESnet at Berkeley Lab, received $62 million in Recovery Act funding.

The Climate 100 project will build on the efforts of DOE's Earth System Grid (ESG), the leading infrastructure for accessing and distributing the impending influx of climate model data.

The ESG was initially developed with funding from DOE's Next Generation Internet Program to address the management and use of extremely large and diverse datasets. It has since become a Center for Enabling Technologies (CET) that provides access to climate data from large scale simulations, metadata information about the models and available datasets, as well as analysis and information tools. The ESG-CET is supported by the DOE's Scientific Discovery through Advanced Computing (SciDAC) program and led by Dean Williams of the Lawrence Livermore National Laboratory. Williams also serves as co-principal investigator on the Climate 100 project.

As an example of the scope of the datasets and user demand, the Program for Climate Model Diagnosis and Intercomparison at Lawrence Livermore National Laboratory has archived climate modeling data from around the world. Known as the World Climate Research Program Coupled Model Intercomparison Project (CMIP3), this open archive now contains more than 35 terabytes of data and is accessed by more than 1,200 users. However, the next-generation archive at Livermore is expected to contain at least 650 terabytes, and the larger distributed worldwide archive will be between 6 to 10 petabytes.

About Computing Sciences at Berkeley Lab

High performance computing plays a critical role in scientific discovery, and researchers increasingly rely on advances in computer science, mathematics, computational science, data science, and large-scale computing and networking to increase our understanding of ourselves, our planet, and our universe. Berkeley Lab’s Computing Sciences Area researches, develops, and deploys new foundations, tools, and technologies to meet these needs and to advance research across a broad range of scientific disciplines.

Founded in 1931 on the belief that the biggest scientific challenges are best addressed by teams, Lawrence Berkeley National Laboratory and its scientists have been recognized with 13 Nobel Prizes. Today, Berkeley Lab researchers develop sustainable energy and environmental solutions, create useful new materials, advance the frontiers of computing, and probe the mysteries of life, matter, and the universe. Scientists from around the world rely on the Lab’s facilities for their own discovery science. Berkeley Lab is a multiprogram national laboratory, managed by the University of California for the U.S. Department of Energy’s Office of Science.

DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit energy.gov/science.