Introduction

Isolating a specific instrument or vocal track from a finished song is a notoriously difficult task in audio engineering, often described as trying to “unbake a cake.” For years, this process required expensive forensic tools or manual phase cancellation. Open-Unmix is an open-source deep learning framework for music source separation that allows researchers, audio engineers, and artists to decompose mixed audio into distinct stems—vocals, drums, bass, and other instruments—using PyTorch. With its reference implementation and pre-trained models, it provides a transparent and reproducible way to achieve high-quality stem extraction without proprietary software.

What Is Open-Unmix?

Open-Unmix is a deep neural network reference implementation for music source separation designed for the PyTorch ecosystem. It is a tool that allows users to separate pop music into four primary stems: vocals, drums, bass, and the remaining instruments. The project is maintained by the SigSep community and is licensed under the MIT License, making it highly accessible for both academic research and commercial application.

The framework is built around a three-layer bidirectional deep LSTM (Long Short-Term Memory) network. This architecture allows the model to analyze audio signals in the time-frequency domain, predicting the magnitude spectrogram of a target source by applying a mask to the input mixture. By training multiple models for each target source, Open-Unmix provides the flexibility to customize training data for specific instruments.

Why Open-Unmix Matters

Before the advent of deep learning, music source separation was limited to basic frequency filtering or crude phase inversion. Open-Unmix fills a critical gap by providing a high-performance, open-source baseline that researchers can use to reproduce results and develop new architectures. Unlike many proprietary tools, Open-Unmix is designed for simplicity and transparency, favoring clear code over monolithic, “black-box” implementations.

The project has gained significant traction, having been awarded 2nd place in the PyTorch Global Summer Hackathon 2020. Its importance lies in its role as a reference implementation; it allows the community to benchmark new models against a known standard. For artists and producers, it removes the barrier to entry for creating high-quality karaoke tracks, samples, or audio restorations, providing professional-grade tools for free.

Key Features

• End-to-End Stem Separation: Decomposes mixed audio tracks into four distinct components: vocals, drums, bass, and other instruments.
• Pre-trained Models: Includes ready-to-use models like umxhq (trained on MUSDB18-HQ for full bandwidth 22050 Hz) and umx (trained on the regular MUSDB18 dataset).
• Bidirectional LSTM Architecture: Uses a three-layer deep LSTM to capture temporal context from both past and future frames, improving separation accuracy.
• Flexible Input Handling: Supports both time-domain signals (waveforms) and pre-computed magnitude spectrograms.
• Differentiable Wiener Filtering: Implements a multichannel generalized Wiener filter (a differentiable version of norbert) to refine estimates and reduce interference between stems.
• Speech Enhancement Model: Includes the umxse model provided by Sony Corporation, specifically trained for speech enhancement on the Voicebank+DEMAND corpus.
• Custom Training Support: Provides a template Dataset class and training scripts, allowing users to train models on their own proprietary audio datasets.
• Broad File Support: Integrates with torchaudio to support various audio formats, with additional support for STEMS export via stempeg.

How Open-Unmix Compares

Open-Unmix is often compared to other AI-driven separators like Spleeter and Demucs. While all three use deep learning, their architectural philosophies differ significantly.

Open-Unmix prioritizes transparency and simplicity. It is the ideal choice for researchers who need a clear, reproducible baseline to test new hypotheses. Spleeter is significantly faster due to its U-Net architecture, making it better for batch processing large libraries of music. Demucs, particularly the newer Hybrid Transformer versions, generally produces higher-fidelity stems with fewer artifacts, especially on complex mixes, but requires significantly more compute power and time.

The trade-off is clear: if you need a research-grade reference implementation that is easy to modify, choose Open-Unmix. If you need raw speed for a commercial application, Spleeter is the better fit. If you are producing a professional remix and need the cleanest possible isolation, Demucs is the industry standard.

Getting Started: Installation

Open-Unmix can be installed via several methods depending on whether you need the full training suite or just the inference tools.

Using pip (Inference Only)

For most users who only want to separate audio files using pre-trained models, the Python package is the fastest route:

pip install openunmix

To increase the number of supported input and output file formats (such as STEMS export), it is recommended to install stempeg:

pip install stempeg

Using Anaconda (Full Suite)

For researchers who need to train their own models, Anaconda is recommended to ensure reproducible results. Create a conda environment using the provided YAML files:

conda env create -f scripts/environment-X.yml

Replace X with your system type: cpu-linux, gpu-linux-cuda10, or cpu-osx.

Using Docker

Open-Unmix provides a Docker container for a zero-config setup. You can separate a local track in a single command:

docker run -v ~/Music/:/data -it faroit/open-unmix-pytorch umx "/data/track1.wav" --outdir /data/track1

How to Use Open-Unmix

The simplest way to use Open-Unmix is through the command-line interface (CLI). Once installed via pip, you can separate a song into stems with a single command:

umx input_file.wav --model umxhq

This command uses the umxhq model, which is the high-quality version trained on MUSDB18-HQ. The tool will automatically download the pre-trained weights and process the audio, saving the separated stems (vocals, drums, bass, other) as individual WAV files in an output directory.

For developers integrating Open-Unmix into a Python application, the framework provides a high-level API. You can load a separator and process audio tensors directly:

import torch
from openunmix import separate

# Load the pre-trained separator
separator = torch.hub.load('sigsep/open-unmix-pytorch', 'umxhq')

# Separate audio (audio should be a torch.Tensor of shape (channels, length))
estimates = separator(audio)

Code Examples

Below are examples of how to use Open-Unmix for different separation scenarios, pulled from the repository’s implementation details.

Example 1: Basic Separation via Python API

This example shows how to load a pre-trained model and separate a mixture of audio into stems using the separate.predict method.

from openunmix import separate

# Use the predict method for a one-step process: loading, preprocessing, and separation
estimates = separate.predict("path/to/audio_file.wav")

Example 2: Advanced Separation with Custom Targets

If you only need specific stems (e.g., just vocals and accompaniment), you can specify the targets using the CLI:

umx input_file.wav --targets vocals --residual

The --residual flag tells Open-Unmix to compute a residual target, effectively creating a “vocals vs. everything else” separation.

Example 3: Training a New Model

To train a model on a custom dataset, you can use the train.py script. This example shows how to start training for a specific target, such as vocals, using a root directory containing your stems:

python train.py --root path/to/musdb18 --target vocals

Real-World Use Cases

Open-Unmix is a versatile tool that shines in several specific professional and academic scenarios:

• Sample Creation for Producers: A hip-hop producer can use Open-Unmix to isolate a drum break from an old vinyl record, creating a clean sample for use in a new track without the interference of other instruments.
• Karaoke and Backing Track Generation: Content creators can generate high-quality instrumental versions of songs by separating the vocals and saving only the “other” and “bass/drums” stems.
• Academic Audio Research: Researchers in signal processing can use Open-Unmix as a baseline to test new neural network architectures for source separation, leveraging the provided MUSDB18 dataset integration.
• Audio Restoration: Audio engineers can isolate specific problematic frequencies or elements in a legacy recording to apply targeted EQ or compression without affecting the rest of the mix.
• umaxse for Speech Enhancement: Using the umxse model, users can clean up noisy voice recordings, removing background noise from a podcast or interview recording.

Contributing to Open-Unmix

Open-Unmix is a community-driven project. Contributions are encouraged to improve the model’s generalization and reduce artifacts. To contribute, follow the standard GitHub flow: fork the repository, create a feature branch, and submit a pull request. The project maintains a CONTRIBUTING.md file that outlines the specific guidelines for submitting bug fixes and feature requests.

The project also encourages the use of GitHub Issues to report bugs or request technical support. If you are a researcher, citing the original Open-Unmix paper in your work is the best way to support the project’s academic standing.

Community and Support

Open-Unmix is part of a broader ecosystem of music separation tools. Official support and community interaction happen primarily through GitHub Discussions and the Gitter chat. For those seeking deeper technical help, the Google Groups forum is also available.

The project’s documentation is hosted on GitHub, and for those who want to try the model without installing anything, a Google Colab notebook is provided in the README, allowing for immediate testing in the cloud.

Conclusion

Open-Unmix for PyTorch is an essential tool for anyone serious about music source separation. By providing a transparent, open-source reference implementation, it has lowered the barrier to entry for researchers and artists alike. While newer models like Demucs may offer slightly higher fidelity in some complex cases, Open-Unmix remains the gold standard for reproducibility and simplicity in the PyTorch ecosystem.

Whether you are building a custom AI audio tool, conducting academic research, or simply need to isolate a vocal track for a remix, Open-Unmix provides the professional-grade infrastructure needed to achieve these results. Star the repo, try the quickstart, and join the community to help push the boundaries of audio processing.

Resources

• Open-Unmix PyTorch GitHub Repository
• MUSDB18 Dataset Documentation
• Open-Unmix Contributing Guidelines