Introduction

Building high-quality visual datasets is often the most tedious part of any computer vision project. Whether you are training a self-driving car or a medical imaging AI, the bottleneck is rarely the model architecture, but the availability of accurately labeled data. CVAT (Computer Vision Annotation Tool) is a professional-grade, open-source platform that solves this by providing a comprehensive suite of tools for annotating images, videos, and 3D point clouds, with over 16k GitHub stars and a massive global community of developers.

What Is CVAT?

CVAT is a web-based, open-source annotation platform that enables users to label images, videos, and 3D point clouds for training computer vision models. Written primarily in Python and TypeScript, it is licensed under the MIT License, allowing teams to self-host the tool to maintain full control over their sensitive data. It serves as the foundation for both CVAT Online and CVAT Enterprise, providing a production-ready environment for building object detection, semantic segmentation, and pose estimation datasets.

Why CVAT Matters

Before CVAT, many teams relied on fragmented, single-purpose tools that required manual exporting and importing of data between different stages of the ML pipeline. CVAT integrates the entire labeling workflow into a single web interface, supporting multi-user collaboration and complex data types like video tracking and 3D cuboids.

The platform’s significance lies in its ability to scale. With millions of Docker pulls and adoption by research and production AI teams worldwide, it has become the industry standard for open-source labeling. By allowing teams to run the software entirely within their own infrastructure, it eliminates the privacy concerns associated with cloud-only labeling services.

Key Features

• AI-Powered Annotation: Connect your own ML models via serverless functions to perform automatic detection, segmentation, and tracking, reducing manual effort by up to 10x.
• 3D Point Cloud Support: Annotate 3D data using cuboids and semantic segmentation, making it essential for LiDAR-based perception pipelines in autonomous driving.
• Video Interpolation: Automatically calculate the position of bounding boxes between keyframes, allowing annotators to label only a few frames and let the tool handle the rest.
• Team Collaboration: Built-in user roles (Admin, User, Annotator, Observer) and task assignment workflows ensure consistent quality across large teams.
• Comprehensive Shape Tools: Support for a wide array of annotation types, including polygons, polylines, points, skeletons, and 3D cuboids.
• Dataset Management: Integrated tools for importing and exporting datasets in various popular formats, ensuring seamless integration with ML frameworks.
• Developer-Friendly APIs: Robust SDKs and APIs that allow for the automation of task creation and data management.
• Self-Hosted Infrastructure: Full control over data privacy and security by running the tool on your own servers via Docker.

How CVAT Compares

CVAT is often compared to other popular labeling tools like Roboflow and Supervisely. While commercial platforms often provide a more polished onboarding experience, CVAT’s open-source nature and self-hosting capability make it the superior choice for teams with strict data privacy requirements or those who need deep customization.

The primary tradeoff is the learning curve. CVAT is a professional tool with a dense interface that can be intimidating for beginners. However, for production-scale CV projects, the reliability and feature set of CVAT generally outweigh the simplicity of rapid prototyping tools.

Getting Started: Installation

CVAT is designed to be deployed via Docker, which is the recommended method for all environments. This ensures that all dependencies, including the database and Redis cache, are bundled together.

Docker Installation (Ubuntu/Linux)

Open a terminal and run the following commands to clone the repository and start the containers:

git clone https://github.com/cvat-ai/cvat
cd cvat
docker compose up -d

Windows Installation (WSL2)

CVAT requires WSL2 for Windows users. Ensure you have Docker Desktop installed and configured to use the WSL2 backend.

1. Install WSL2 and a Linux distribution (e.g., Ubuntu) from the Microsoft Store.
2. Open the Linux terminal and clone the repo: git clone https://github.com/cvat-ai/cvat.
3. Run docker compose up -d to start the CVAT instance.

Development Environment Setup

For those looking to modify the source code, a full development environment is required. This involves installing Node.js 20, Python 3.10, and various system dependencies like libldap2-dev and libgeos-dev.

# Example for Ubuntu
sudo apt-get update && sudo apt-get install -y build-essential curl git python3-dev python3-pip python3-venv python3-tk libldap2-dev libsasl2-dev libgeos-dev cargo
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo bash - 
sudo apt-get install -y nodejs

How to Use CVAT

Once CVAT is running, access the tool via localhost:8080 in your browser. The basic workflow follows a project-based hierarchy: Project → Task → Job.

First, create a Project to define the labels and attributes you need for your dataset. Then, create a Task by uploading your raw data (images or video files). Finally, the task is divided into Jobs, which can be assigned to individual annotators.

In the editor, you can use the bounding box tool to draw shapes, use the interpolation feature to track objects across frames, and use the Ctrl + S shortcut to save your work. The entire process is designed to maximize the attraverso-put of the annotator while minimizing repetitive clicks.

Code Examples

CVAT provides a Python SDK and a REST API to automate the creation of tasks and the management of annotations. This is useful for teams that have thousands of images and want to avoid manual uploads.

The following example demonstrates how to create a new annotation task using the CVAT Python SDK:

from cvat_sdk import Client

client = Client("http://localhost:8080")
client.login("username", "password")

task = client.tasks.create(
    name="Example Task",
    labels=[{"name": "Car", "attributes": [{"name": "Color", "type": "text"}]}]
)
print(f"Task created with ID: {task.id}")

This script allows you to programmatically define labels and and upload data, which is a critical step for scaling your data pipeline.

Real-World Use Cases

CVAT is used across various industries where high-precision visual data is required for AI training.

• Autonomous Driving: Engineers use CVAT to label 3D LiDAR point clouds with cuboids to help vehicles recognize pedestrians, other cars, and road boundaries in 3D space.
• Medical Imaging: Researchers use the polygon tool for semantic segmentation of tumors or organs in MRI and CT scans, ensuring high-precision boundaries for diagnostic AI.
• Surveillance and Security: Security teams use video interpolation and tracking tools to label objects across hundreds of frames, creating datasets for behavioral analysis AI.
• Robotics: Robotics developers use CVAT to create training data for pick-and-place robots, labeling objects in a warehouse environment to improve spatial awareness.

Contributing to CVAT

CVAT is a community-driven project. If you find a bug or want to add a feature, the team encourages contributions via GitHub. The project follows a standard open-source flow: fork the repository, create a feature branch, and submit a pull request.

New contributors should start by looking for issues labeled good first issue to get acclimated to the project’s large codebase. The project also maintains a strict code of conduct to ensure a professional and welcoming environment for all developers.

Community and Support

CVAT has one of the most active communities in the computer vision space. Official support and discussions take place primarily on GitHub Discussions and the official CVAT.ai documentation site.

Users can also join the CVAT Discord server for real-time help and troubleshooting. The project’s activity level is very high, with frequent commits and a large number of open issues and discussions, indicating a well-maintained and production-ready tool.

Conclusion

CVAT is the gold standard for open-source computer vision annotation. For teams that need a professional, self-hosted tool that supports images, videos, and 3D data, it is the most complete solution available. While the interface has a learning curve, the power and flexibility it provides for production-scale datasets are unmatched.

If you are building a vision AI project and want full control over your data, we recommend starting with the CVAT Community edition. Star the repo, try the quickstart, and join the community to start building high-quality datasets.

Resources

• CVAT GitHub Repository
• Official CVAT Documentation
• CVAT Academy for Learning