Introduction

Medical image segmentation often requires tedious manual tuning of network architectures, preprocessing pipelines, and hyperparameters, which can be a significant bottleneck for researchers and clinicians. nnU-Net is an open-source semantic segmentation framework that solves this by automatically adapting its entire pipeline to any given dataset, with over 8,500 GitHub stars. It replaces the need for manual trial-and-error by turning dataset properties into a reproducible configuration process, making it the gold standard for biomedical image analysis.

What Is nnU-Net?

nnU-Net is a self-configuring deep learning framework that automatically configures a U-Net-based segmentation pipeline for biomedical images. Developed by the Applied Computer Vision Lab (ACVL) of Helmholtz Imaging and the Division of Medical Image Computing at the German Cancer Research Center (DKFZ), it is written in Python and licensed under the Apache License 2.0.

Unlike traditional segmentation tools, nnU-Net does not require the user to manually specify the patch size, batch size, or network topology. Instead, it analyzes the “fingerprint” of the dataset—including image sizes, voxel spacings, and intensity distributions—to determine the optimal settings for training and inference.

Why nnU-Net Matters

In the past, creating a high-performing segmentation model for a new medical dataset required deep expertise in deep learning and weeks of manual optimization. Because medical datasets vary wildly in terms of modality (CT, MRI), dimensionality (2D vs 3D), and anisotropy (different resolutions along different axes), a pipeline that works for the brain may fail for the liver.

nnU-Net matters because it democratizes high-performance medical imaging AI. By providing a standardized, data-set agnostic recipe, it allows clinicians and researchers without extensive AI expertise to achieve state-of-the-art results. It has become a primary benchmark against which new architectures are measured, as it consistently wins medical imaging challenges across diverse anatomical regions.

Key Features

• Automated Dataset Fingerprinting: The framework extracts a set of dataset-specific properties (image sizes, voxel spacings, intensity information) to drive all subsequent pipeline decisions.
• Self-Configuring Pipeline: Automatically determines the optimal patch size, batch size, and network topology based on the dataset fingerprint and available hardware memory.
• Multi-Dimensional Support: Native support for both 2D and 3D data, including specialized handling for anisotropic data where spatial resolutions differ across axes.
• U-Net Configurations: Generates multiple configurations including a 2D U-Net, a 3D full-resolution U-Net, and a 3D U-Net cascade (where a low-res model refines a coarse map).
• Robust Data Augmentation: Includes a comprehensive suite of built-in augmentations specifically tailored for medical imaging to improve model generalization.
• Cross-Platform Compatibility: Supports Linux (primary target), Windows, and macOS, with optimized support for NVIDIA GPUs via CUDA and Apple Silicon via MPS.
• Multi-GPU Training: nnU-Net v2 introduces native support for training across multiple GPUs to accelerate the process for large 3D volumes.
• Region-Based Formulation: v2 also implements region-based segmentation with sigmoid activation, providing more flexibility in how labels are handled.

How nnU-Net Compares

While MONAI is a comprehensive library of medical imaging components (transforms, networks, loaders), nnU-Net is a complete, opinionated pipeline. The primary differentiator is that nnU-Net removes the decision-making process for the user. Where MONAI provides the building blocks to build a pipeline, nnU-Net provides the finished pipeline that automatically adapts to the data.

V-Net, while pioneering 3D segmentation, is a specific architecture. nnU-Net incorporates the lessons of V-Net and U-Net but wraps them an automation layer that ensures the architecture is used correctly for the specific voxel spacing and image size of the dataset.

Getting Started: Installation

nnU-Net requires a GPU for training. For inference, a GPU with at least 4 GB VRAM is recommended, while training requires at least 10 GB VRAM.

Standard Installation

First, install PyTorch according to your hardware (CUDA for NVIDIA, MPS for Apple Silicon, or CPU). Do not install nnU-Net before PyTorch is in place.

pip install nnunetv2

Editable Installation for Development

If you intend to modify the source code or add custom trainers, use an editable install:

git clone https://github.com/MIC-DKFZ/nnUNet.git
cd nnUNet
pip install -e .

Prerequisites and Environment

nnU-Net requires Python 3.10 or newer. It is strongly recommended to use a virtual environment (conda or venv). For Intel-based macOS users, use the following command to pin compatible versions:

pip install "nnunetv2[intel_macos]"

How to Use nnU-Net

The workflow in nnU-Net is strictly defined to ensure reproducibility. It consists of three main stages: data preparation, preprocessing, and training.

1. Data Preparation: Organize your images and labels in the nnUNet_raw folder. Each dataset must follow a specific naming convention (e.g., Dataset001_Brain) and include a dataset.json file describing the modalities and labels.

2. Preprocessing: Run the planning and preprocessing command. nnU-Net will analyze the dataset fingerprint and create the training plans.

nnUNetv2_plan_and_preprocess -d Dataset001_Brain

3. Training: Start the training process for a specific configuration (e.g., 3d_fullres). You must specify the fold for 5-fold cross-validation.

nnUNetv2_train -d Dataset001_Brain -c 3d_fullres -f 0

4. Inference: Once trained, use the predict command to generate segmentations for new images.

nnUNetv2_predict -i /path/to/input -o /path/to/output -d Dataset001_Brain -c 3d_fullres -f 0

Code Examples

While nnU-Net is primarily a CLI tool, you can integrate it into Python scripts. The following examples demonstrate how to interact with the framework’s core logic.

Dataset Fingerprinting

The fingerprinting process is what allows nnU-Net to adapt. It extracts median shapes and voxel spacings from the images.

# Example of how nnU-Net internally handles fingerprinting
from nnunetv2.dataset_fingerprint import DatasetFingerprint

# This is typically handled by the CLI, but can be accessed via API
fingerprint = DatasetFingerprint(dataset_id=1)
print(f"Median Image Shape: {fingerprint.median_shape}")
print(f"Median Spacing: {fingerprint.median_spacing}")

Running Inference via Python

You can use the nnUNetv2_predict logic within a larger Python pipeline for medical imaging analysis.

# Simplified example of calling the predictor
from nnunetv2.inference.predict_from_raw_data import nnUNetv2_predict

# Define input and output paths
input_folder = "/path/to/images"
output_folder = "/path/to/results"

nUNetv2_predict(
    input_folder=input_folder,
    output_folder=output_folder,
    dataset_id=1,
    configuration="3d_fullres",
    folds=[0],
    device=torch.device("cuda")
)

Advanced Configuration

nnU-Net requires three specific environment variables to be set for it to function. These variables tell the framework where to store raw data, preprocessed data, and the final trained models.

# Add these to your .bashrc or .zshrc for persistence
export nnUNet_raw="/path/to/nnUNet_raw"
export nnUNet_preprocessed="/path/to/nnUNet_preprocessed"
export nnUNet_results="/path/to/nnUNet_results"

Additionally, you can optimize data augmentation performance by setting the nnUNet_n_proc_DA variable, which controls the number of CPU processes used for data augmentation. A recommended value is 12-16 for high-end GPUs like the RTX 3090/4090.

Real-World Use Cases

nnU-Net is widely used in clinical research and competition settings where accuracy is the primary goal.

• Organ Segmentation: A radiologist can use nnU-Net to automatically segment the liver, kidneys, and spleen from CT scans, reducing the time spent on manual contouring from hours to seconds.
• Tumor Volume Tracking: An oncologist can train a model to segment brain glioblastomas in multi-parametric MRI scans (e.g., BraTS dataset), allowing for precise tracking of tumor growth or shrinkage during chemotherapy.
• T1-Weighted MRI Spine Segmentation: Researchers can use the 3D full-resolution configuration to segment individual vertebrae in the lumbar spine, handling the high anisotropy of the scans.
• Pancreas Segmentation: Because the pancreas is a small, highly variable organ, nnU-Net’s automated patch size and batch size optimization is critical for capturing the organ’s boundaries accurately.

Contributing to nnU-Net

nnU-Net is an open-source project maintained by the MIC-DKFZ organization. Contributions are welcome through the standard GitHub flow. Users should first search the existing issues to see if a bug has been reported or if a feature request has been made.

To contribute, developers can submit pull requests for bug fixes or new trainer classes. The project follows a strict code of conduct to ensure a professional environment. Bug reports should be detailed, including the dataset fingerprint and the hardware configuration used during training.

Community and Support

The primary hub for support is the GitHub Discussions forum, where users can ask questions about dataset preparation and installation errors. The official documentation is hosted on GitHub in the /documentation folder, providing a detailed explanation of the concepts and rationale behind the framework.

The community is active, with thousands of users across the medical imaging AI community. For those looking for deeper integration, the MONAI framework provides tutorials and runners for nnU-Net, bridging the gap between the modular components of MONAI and the automated pipeline of nnU-Net.

Conclusion

nnU-Net is the definitive tool for medical image segmentation, providing a robust, automated pipeline that removes the guesswork from deep learning. For researchers who need a state-of-the-art baseline, it is the right choice. For clinicians who want to deploy AI in their workflow without becoming deep learning experts, it is an essential tool.

While it is computationally expensive—requiring significant GPU VRAM and CPU cores—the trade-off is a level of performance and reproducibility that is nearly impossible to achieve through manual tuning. Star the repo, try the quickstart, and join the community to start automating your medical imaging analysis.

Resources

• nnU-Net Official GitHub Repository
• nnU-Net Documentation and Rationale
• nnU-Net Documentation and Rationale