# Optimization

Optimization is central to scientific computing as well as scientific machine learning. Both the basic methodology as well as application of optimization algorithms are widespread. Berkeley Lab researchers specialize in stochastic optimization, stochastic second-order methods, and constrained machine learning optimization methods to tackle issues in climate science, quantum computing, high energy physics, advanced energy systems, and water treatment systems, with many of our most recent advances motivated by machine learning workflows. Depending on the application, we employ a broad range of optimization algorithms, from deterministic to stochastic, from first-order to second-order to hybrid, and from problems where the entire problem can be stored in RAM to black-box methods where the problem is accessed implicitly only via expensive simulations. Our work also has strong connections with sampling methods, in particular in scientific machine learning, where the stochasticity of stochastic gradient methods can lead to implicit sampling.

## Projects

### HGDL for Hybrid Global Deflated Local Optimization

HGDL is an optimization algorithm specialized in finding not only one but a diverse set of optima, alleviating challenges of non-uniqueness that are common in modern applications such as inversion problems and training of machine learning models. HGDL is customized for distributed high-performance computing; all workers can be distributed across as many nodes or cores. All local optimizations will then be executed in parallel. As solutions are found, they are deflated which effectively removes those optima from the function, so that they cannot be re-identified by subsequent local searches. **Contact: Marcus Noack**

### Normalizing Flows for Statistical Data Analysis

We use Normalizing Flows to develop fast Bayesian statistical analysis methods for scientific data analysis that can be applied to a wide range of scientific domains and problems. The methods can be used for posterior sampling and global optimization applications, with or without gradient information. Recent examples are DLMC (Deterministic Langevin Monte Carlo) and Preconditioned Monte Carlo (PocoMC). **Contact: Uros Seljak**

### Institute for the Design of Advanced Energy Systems (IDAES)

The IDAES integrated platform (IDAES-IP) brings the most advanced modeling and optimization capabilities to challenges around the reliable, environmentally sustainable, and cost-efficient transformation and decarbonization of the world’s energy systems. IDAES utilizes state-of-the-art equation-oriented optimization solvers and algorithms to enable the design, optimization, and operation of complex, innovative steady state and dynamic processes. **Contact: Dan Gunter (Gunter on the Web)**

### NAWI Water Treatment Model Development (WaterTAP)

Water treatment, as a sophisticated environmental and chemical engineering practice, designs physical/chemical/biological processes to produce clean water. We provide computational and modeling solutions to optimize the performance, energy use, and economic cost of existing and developing water treatment processes and infrastructures. Conventional linear and nonlinear programming are applied to theory- and data-informed equation systems describing the engineering systems at different scales. We ultimately deliver the optimization efficacy as user-friendly and open-source software. We explore the potential advantages of novel ML and AI algorithms to complement conventional numerical optimization approaches, tackling complexity and dynamics challenges in broad water systems. **Contact: Dan Gunter (Gunter on the Web)**

### Produced Water Application for Beneficial Reuse, Environmental Impact and Treatment Optimization (PARETO)

The Produced Water Application for Beneficial Reuse, Environmental Impact and Treatment Optimization (PARETO) is specifically designed for produced water management and beneficial reuse. The major deliverable of this project will be an open-source, optimization-based, downloadable and executable produced water decision-support application, PARETO, that can be run by upstream operators, midstream companies, technology providers, water end users, research organizations, and regulators. **Contact: Dan Gunter (Gunter on the Web)**

## News

## Berkeley Lab’s PARETO Wins Award for Engineering Innovation

The Produced Water Optimization Program (PARETO) framework, a collaboration between Lawrence Berkeley National Laboratory and the National Energy Technology Laboratory, was named a winner in Hart Energy’s 2022 Special Meritorious Awards for Engineering Innovation for its water management capabilities. Read More »

## HYPPO: Leveraging Prediction Uncertainty to Optimize Deep Learning Models for Science

With a growing need for optimization tools that can enhance deep learning models to improve predictive capabilities and accelerate time-consuming computer simulations, a Berkeley Lab team developed HYPPO, an open-source software tool for hyperparameter optimization of deep neural networks. Read More »

## IDAES Honored with R&D100 Award

The U.S. Department of Energy’s (DOE’s) Institute for the Design of Advanced Energy Systems (IDAES) is the winner of a prestigious 2020 R&D100 award, which recognizes the developers of the 100 most technologically significant products introduced into the marketplace in the last year. Read More »

## CRD’s Deb Agarwal Named to Committee to Help Shape California State Water Data Structure

Deb Agarwal, head of the Data Science and Technology Department in Berkeley Lab's Computational Research Division, is one of 11 members named to the inaugural steering committee of the California Water Data Consortium. Read More »

## IDAES Process Systems Engineering Software Now Open Source

IDAES has released the first open-source version of its next-generation computational framework and model library, created to optimize the design of process systems engineering solutions used to model advanced energy systems. Read More »