A-Z Index | Directory | Careers

ESnet Plays Key Role in Groundbreaking ARIES 'Digital Twin' Demonstration

NREL-led Collaboration Connects Distributed Power Grid Assets at National Labs via Low-Jitter Network

February 10, 2022

This news release was originally published by the National Renewable Energy Laboratory.

We can currently link models and simulations across the national laboratory complex, but imagine connecting these models with equipment, devices, and scientists so they can work together as if side by side instead of thousands of miles apart. In a future like this, researchers could run experiments that leverage thousands of devices to model the millions of devices that make up our increasingly complex energy systems.

natl labs map with ESnet2

With such resources, labs could simulate, for example, the electric grid of California and validate control strategies to best serve critical loads during outages caused by wildfires.

The National Renewable Energy Laboratory's (NREL's) Advanced Research on Integrated Energy Systems (ARIES) platform is making that vision a reality, one step at a time. A recent demonstration built upon previous efforts to connect labs and make more detailed energy research possible.

"The vision to connect lab assets via a high-speed network started with the RT-SuperLab project," said Juan Torres, NREL associate laboratory director for Energy Systems Integration. "The ARIES team at NREL and our DOE and lab partners are building on that project. The goal is to create a network that can support large-scale energy system research and experimentation. This is critical in order to get in front of the challenges – and develop solutions – for a clean energy future."

The SuperLab that Started it All

In September 2017, researchers performed a demonstration called the Global RealTime (RT) SuperLab. A total of eight laboratories across two continents used a Digital Real Time Simulation (DRTS) system connected over the internet for the demonstration.

The first SuperLab experiment was successful, but it uncovered a problem: in a word, latency. Latency is a measure of delay, and in this case, it was in transferring data using a standard internet connection across thousands of miles. Not only was latency in this experiment sometimes large, but it was also unpredictable. That variability made it difficult for researchers to conduct a meaningful experiment.

"Consistent latency is critical because without it we can't develop filters that enable large systems to talk with each other," said Rob Hovsapian, a research advisor with NREL.

The Latency Fix: The Office of Science's ESnet

After the first SuperLab experiment, researchers began the search for solutions that would correct for latency and allow labs to run experiments together as if side by side.

That is where the U.S. Department of Energy's (DOE's) Office of Science's network facility, the Energy Sciences Network (ESnet), came in. ESnet is a high-performance unclassified national network built and dedicated to supporting scientific research. ESnet has made critical contributions not only for the national laboratories but also internationally, including supporting the Large Hadron Collider, the world's largest and most powerful particle accelerator.

Combined with the On-demand Secure Circuits and Advance Reservation System (OSCARS) circuit, an advanced software system for booking time and resources on high-speed science networks, researchers could ensure consistent, low-latency performance during experiments.

"OSCARS deftly cuts through scientific network scheduling complexity and can reserve as well as automatically configure a reliable, customized set of network connections in a matter of minutes, not weeks," said Inder Monga, division director at ESnet. "In addition to science applications, OSCARS provides network operators the ability to dynamically provision dedicated backups, should their network experience unforeseen outages."

Putting the Theory into Practice

In early December 2021, NREL's ARIES researchers teamed up with the Pacific Northwest National Laboratory (PNNL) and ESnet staff to demonstrate these capabilities during their live demonstration of a Cordova, Alaska, microgrid modeling experiment.

The December multi-laboratory demonstration showed that advanced control systems in the Cordova microgrid could allow it to maintain power to critical resources like the hospital and the airport during an extreme weather event and loss of some of its hydropower resources. During this tightly coupled experiment, NREL simulated the Cordova microgrid while PNNL simulated the advanced control systems.

This first-of-a-kind demonstration brought together the best of the laboratory complex, showcasing the ability to integrate expertise and capabilities at multiple DOE laboratories to address unique challenges that no one laboratory could address on its own.

ESnet provided a reliable, low-latency connection so that research equipment at the two laboratories could exchange frequent command and control information. The round-trip latency between NREL and PNNL was 24 milliseconds, with a variance of approximately 0.02 milliseconds. The average latency in the original SuperLab demonstration was about 27 milliseconds with a variance of about 11.5 milliseconds. The very low latency variance was vital to the research success and made exchange of command-and-control information between the two laboratories in the December demonstration nearly deterministic.

More than 90 energy experts, including DOE representatives from the Office of Science, the Office of Nuclear Energy, and the Office of Energy Efficiency and Renewable Energy, attended this demonstration.

Stepping into the Future of Energy Experimentation

Future energy systems are increasing in complexity, demanding experimentation that reflects those conditions. On the heels of this successful demonstration, NREL's ARIES researchers and laboratory partners are already working toward the next step. The team is looking to connect six national laboratories for experiments that use and model millions of interconnected devices. These experiments would be called "SuperLab 2.0."

This would allow researchers to run simulations like the Cordova microgrid, but at a scale that reflects the complexity of a regional or national energy system.

Such an advanced research network could better identify solutions for city- and region-level problems. These experiments might include outages caused by extreme weather events or a cyberattack. It could also help energy systems managers who are adopting new technologies that rely on interconnected devices, so they can be assured that there are proven solutions to challenges that may arise.

"As communities, utilities, and other stakeholders work together to transition to a clean energy future, the research capabilities demonstrated will reduce the risk in this transition and ensure that our future energy system is affordable, reliable, resilient, sustainable, and secure," said Kevin Lynn, director of grid modernization in DOE'stake  Office of Energy Efficiency and Renewable Energy.

For further information on this demonstration, please contact Rob Hovsapian or Steve Hammond with the National Renewable Energy Laboratory.

About Computing Sciences at Berkeley Lab

High performance computing plays a critical role in scientific discovery, and researchers increasingly rely on advances in computer science, mathematics, computational science, data science, and large-scale computing and networking to increase our understanding of ourselves, our planet, and our universe. Berkeley Lab’s Computing Sciences Area researches, develops, and deploys new foundations, tools, and technologies to meet these needs and to advance research across a broad range of scientific disciplines.

Founded in 1931 on the belief that the biggest scientific challenges are best addressed by teams, Lawrence Berkeley National Laboratory and its scientists have been recognized with 13 Nobel Prizes. Today, Berkeley Lab researchers develop sustainable energy and environmental solutions, create useful new materials, advance the frontiers of computing, and probe the mysteries of life, matter, and the universe. Scientists from around the world rely on the Lab’s facilities for their own discovery science. Berkeley Lab is a multiprogram national laboratory, managed by the University of California for the U.S. Department of Energy’s Office of Science.

DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit energy.gov/science.