Software for Workflows and Data

Software has become a critical component of the scientific ecosystem to support workflows and data. Berkeley Lab has developed software services to support scientific workflows and data from various scientific domains. This software includes workflow tools, provenance collections, data management, data formats, data analytics and visualization tools.

Projects

Science Capsule

Science Capsules enable reproducibility, sharing, and reuse of next-generation data analysis workflows from experimental, observational, and simulation data. We are designing and developing a Science Capsule abstraction that captures the processing and data life cycle across machines, organizations, people, and science domains. Our approach combines user engagement methods from social sciences with innovations in workflow and data management. Contact: Lavanya Ramakrishnan

SCIRA: Scalable Collaborative Interactive Reproducible Analytics

We are developing abstractions to support robust, scalable and reproducible data analysis workflows on next-generation High Performance Computing (HPC) systems. Contact: Lavanya Ramakrishnan

Science Search

In Science Search, we are using machine learning to generate automated metadata that enables search on the data.Contact: Lavanya Ramakrishnan

GAUDI / Athena

GAUDI is a data processing, analysis and simulation C++ framework that is used by a number of High Energy Physics experiments, notably LHCb, ATLAS, DayaBay, and Minerva. Athena is the ATLAS specific extension to GAUDI with a specialized python driven frontend configuration layer.Contact: Charles Leggett, (Leggett on the Web)

Raythena / HPX: HEP workflows for HPCs,/strong>

Raythena is a vertically integrated application scheduler for High Energy Physics targeting distributed HPC systems which matches available resources to application requirements. It integrates the Ray distributed workflow manager and the Athena framework from the ATLAS experiment at CERN.Contact: Paolo Calafiura, (Leggett on the Web)