Zenesis: A ‘ChatGPT’ for Scientific Image Analysis

Software Advancement

Berkeley Lab researchers developed Zenesis, an interactive, no-code computer vision platform that provides a “ChatGPT-like” experience for scientific imaging. Zenesis lets domain scientists rapidly isolate and segment complex features, such as pores or crystals, using natural-language queries instead of tedious manual tracing. It operates directly on raw, “non-AI-ready” scientific data with more than 98% accuracy. No model training is needed for initial scientific exploration. By removing the need for manual annotation and coding, Zenesis turns raw images into clean, analysis-ready results in minutes. Because it works without hand-labeled examples, it also automatically generates the high-quality training data that raw scientific data usually lacks, making further, specialized AI development faster and easier when needed.

Background

While mainstream AI models can identify everyday objects in photographs simply by following text instructions (prompt-based) and without needing prior training on those specific items (zero-shot), they consistently fail on specialized scientific data. Scientific images feature unique microstructures, complex noise profiles, and grayscale intensities entirely absent from everyday photos, so preparing them for analysis creates massive “data readiness bottlenecks” that force researchers to spend countless hours manually annotating features. There is an urgent need in the scientific community for tools that can bypass this bottleneck, enabling researchers to efficiently analyze diverse and challenging imaging modalities (types) like Focused Ion Beam Scanning Electron Microscopy (FIB-SEM), cryo-TEM, and micro-CT to accelerate downstream tasks like porosity and grain size analysis.

Breakdown

Zenesis overcomes these challenges not by just training a bigger model, but by making existing foundation models “see” scientific images. 

At its core is a scientific structure-sensitive intelligence that adapts its approach based on what it is looking for. For compact features like crystals or dense particles surrounded by messy background, Zenesis first locates the feature from the text query and then outlines it precisely. For trickier structures where boxes don’t work — like long, branching cracks in a novel material or hundreds of tiny, tightly packed cell nuclei — it switches to a different mode that segments the pattern directly, without trying to force it into a box.

To make models trained on everyday photos work on raw, noisy scientific data without retraining, Zenesis integrates this intelligence layer with GroundingDINO, an AI model that translates a researcher’s text instructions into initial boundary boxes around requested features. These regions of interest are then fed into SAM, a Segment Anything Model, which tightens them into highly precise, pixel-level outlines known as segmentation masks. Zenesis automatically cleans up duplicate detections in cluttered images, remembers what it saw in one slice to work faster in the next, and keeps shapes steady through a full 3D volume.

Running on HPC systems with an interactive interface that keeps the human expert in control, it turns hours of manual tracing into minutes of quick review — producing clean results ready for discovery, and training data ready for future AI models without hand-labeling.

Developers

Shubhabrata Mukherjee (Berkeley Lab), Daniela Ushizima (Berkeley Lab)

Publication

Foundation Models for Zero-Shot Segmentation of Scientific Images without AI-Ready Data

Funding

• DOE ASCR: Autonomous Solutions for Computational Research with Immersive Browsing & Exploration (ASCRIBE) and Center for Advanced Mathematics for Energy Research Applications (CAMERA)
• DOE ASCR and BER: Center for Restoration of Soil Carbon by Precision Biological Strategies (Restor-C)
• DOE ASCR and BES: Center for Ionomer-based Water Electrolysis (CIWE)