On October 12 the Energy Sciences Network (ESnet), the U.S. Department of Energy (DOE)’s dedicated high-speed network for science, kicked off a new era in data transmission for research with the unveiling of ESnet6. As the circulatory system for data transfer across the national lab complex and all 28 DOE SC user facilities, ESnet6 represents a step toward the future of research across the system and beyond. At the upcoming International Conference for High Performance Computing, Networking, Storage, and Analysis (SC22) in Dallas, ESnet Director Inder Monga will present the features and future of ESnet6 to give academic, government, and industry leaders details of its capabilities.

Six years in the making, ESnet6 is composed of 15,000 miles of fiber optic cable with an aggregate capacity of 46 terabits per second. The new network was built from the ground up by ESnet and gives staff an unprecedented level of control of the network and its next-generation features.

“For the first time, we have full control from the fiber all the way up to the network devices, and we’ve integrated it as a team, all by ourselves,” said Monga. “It gives us access to large amounts of bandwidth, and of course, we’ll be growing it as traffic needs increase.”

Among the features that make ESnet6 unique is a high-touch packet telemetry system capable of processing telemetry data from up to 300 million packets per second per node. With accurate hardware timestamping, any packet can be located, moved, and potentially manipulated in transit, an improvement on the statistical sampling approach currently found on the commercial market. With this capability, ESnet staff can diagnose performance issues with petabyte-scale data transfers in real time, enabling the researcher to accelerate their time to discovery.

Additionally, ESnet6 includes integrated software automation and orchestration capabilities, built right into the network, allowing configuration to take place automatically across the whole network rather than addressing each task by hand.

This automation is key to the future of integrated research infrastructure, which will increasingly allow data transmission between experimental facilities and supercomputing centers without regular human intervention. This type of integration and automation eases the infrastructure management burden on researchers, shortens the amount of time needed to transmit and analyze experimental data, allows for informed decision-making during experiments, and ultimately accelerates the pace of scientific discovery. According to Monga, the automation built into ESnet6 will improve research interactions across the DOE system.

“As the data intensity of science simulations and the mixing of experimental and observational data increases, we need a substrate that is capable of moving the data around,” he said. “But also, different supercomputing centers have different capabilities, and the network allows access to all those facilities and enables researchers to truly leverage the value each of these facilities brings, so science can happen faster. Our mission is to reduce barriers so that data can go to the right compute, in a resilient manner, so that insights are generated.”

In the end, the success of ESnet6 is directly tied to people: the researchers who use it and the team who worked tirelessly to build it, even during a pandemic that limited in-person teamwork and travel.

“A network like this couldn’t be built without the people behind it,” he said. “Sometimes you get so enamored with the technology and the statistics, the 15,000 miles of fiber and 46 terabytes per second, but so much of this effort was thanks to the ESnet6 team – their dedication, their mission-orientedness, and how they worked really hard to make this happen and be successful. The project came in ahead of schedule and under budget. This facility makes us proud, but it’s the people and their efforts that are actually on display.”

Monga will present as a DOE featured speaker on “How ESnet’s Next-Generation Infrastructure Will Enable Integrated Research Initiative Workflows” at 3:15 p.m. CST Tuesday, November 15, in the DOE booth, #1600.