Inder Monga Introduces ESnet6 at SC22
November 10, 2022
By Elizabeth Ball
On October 12 the Energy Sciences Network (ESnet), the U.S. Department of Energy (DOE)’s dedicated high-speed network for science, kicked off a new era in data transmission for research with the unveiling of ESnet6. As the circulatory system for data transfer across the national lab complex and all 28 DOE SC user facilities, ESnet6 represents a step toward the future of research across the system and beyond. At the upcoming International Conference for High Performance Computing, Networking, Storage, and Analysis (SC22) in Dallas, ESnet Director Inder Monga will present the features and future of ESnet6 to give academic, government, and industry leaders details of its capabilities.
Six years in the making, ESnet6 is composed of 15,000 miles of fiber optic cable with an aggregate capacity of 46 terabits per second. The new network was built from the ground up by ESnet and gives staff an unprecedented level of control of the network and its next-generation features.
“For the first time, we have full control from the fiber all the way up to the network devices, and we’ve integrated it as a team, all by ourselves,” said Monga. “It gives us access to large amounts of bandwidth, and of course, we’ll be growing it as traffic needs increase.”
Among the features that make ESnet6 unique is a high-touch packet telemetry system capable of processing telemetry data from up to 300 million packets per second per node. With accurate hardware timestamping, any packet can be located, moved, and potentially manipulated in transit, an improvement on the statistical sampling approach currently found on the commercial market. With this capability, ESnet staff can diagnose performance issues with petabyte-scale data transfers in real time, enabling the researcher to accelerate their time to discovery.
Additionally, ESnet6 includes integrated software automation and orchestration capabilities, built right into the network, allowing configuration to take place automatically across the whole network rather than addressing each task by hand.
This automation is key to the future of integrated research infrastructure, which will increasingly allow data transmission between experimental facilities and supercomputing centers without regular human intervention. This type of integration and automation eases the infrastructure management burden on researchers, shortens the amount of time needed to transmit and analyze experimental data, allows for informed decision-making during experiments, and ultimately accelerates the pace of scientific discovery. According to Monga, the automation built into ESnet6 will improve research interactions across the DOE system.
“As the data intensity of science simulations and the mixing of experimental and observational data increases, we need a substrate that is capable of moving the data around,” he said. “But also, different supercomputing centers have different capabilities, and the network allows access to all those facilities and enables researchers to truly leverage the value each of these facilities brings, so science can happen faster. Our mission is to reduce barriers so that data can go to the right compute, in a resilient manner, so that insights are generated.”
In the end, the success of ESnet6 is directly tied to people: the researchers who use it and the team who worked tirelessly to build it, even during a pandemic that limited in-person teamwork and travel.
“A network like this couldn’t be built without the people behind it,” he said. “Sometimes you get so enamored with the technology and the statistics, the 15,000 miles of fiber and 46 terabytes per second, but so much of this effort was thanks to the ESnet6 team – their dedication, their mission-orientedness, and how they worked really hard to make this happen and be successful. The project came in ahead of schedule and under budget. This facility makes us proud, but it’s the people and their efforts that are actually on display.”
Monga will present as a DOE featured speaker on “How ESnet’s Next-Generation Infrastructure Will Enable Integrated Research Initiative Workflows” at 3:15 p.m. CST Tuesday, November 15, in the DOE booth, #1600.
About Computing Sciences at Berkeley Lab
High performance computing plays a critical role in scientific discovery. Researchers increasingly rely on advances in computer science, mathematics, computational science, data science, and large-scale computing and networking to increase our understanding of ourselves, our planet, and our universe. Berkeley Lab’s Computing Sciences Area researches, develops, and deploys new foundations, tools, and technologies to meet these needs and to advance research across a broad range of scientific disciplines.
Founded in 1931 on the belief that the biggest scientific challenges are best addressed by teams, Lawrence Berkeley National Laboratory and its scientists have been recognized with 16 Nobel Prizes. Today, Berkeley Lab researchers develop sustainable energy and environmental solutions, create useful new materials, advance the frontiers of computing, and probe the mysteries of life, matter, and the universe. Scientists from around the world rely on the Lab’s facilities for their own discovery science. Berkeley Lab is a multiprogram national laboratory, managed by the University of California for the U.S. Department of Energy’s Office of Science.
DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit energy.gov/science.