Introduction

Labeling 3D point clouds and volumetric data is a notoriously difficult process, often requiring specialized software that is either prohibitively expensive or difficult to integrate into existing workflows. CVAT-CANVAS-3D solves this by providing a dedicated, TypeScript-based canvas for viewing, drawing, and editing 3D annotations within the broader CVAT ecosystem. As part of the open-source Computer Vision Annotation Tool (CVAT), which has earned widespread adoption in the AI community, this module enables developers to build high-quality 3D datasets for autonomous driving, robotics, and industrial inspection.

What Is CVAT-CANVAS-3D?

CVAT-CANVAS-3D is a TypeScript-based module that provides a specialized 3D canvas for the viewing, drawing, and editing of 3D annotations. It is designed to function as a core component of the Computer Vision Annotation Tool (CVAT), allowing users to interact with 3D objects and scenes—such as vehicles, buildings, and landscapes—directly in the browser.

Maintained as part of the open-source CVAT project under the MIT License, the module focuses on providing a high-performance interface for 3D cuboid and shape annotation. It transforms raw 3D data, such as LiDAR point clouds, into structured, meaningful information that can be used to train machine learning models for spatial analysis and autonomous navigation.

Why CVAT-CANVAS-3D Matters

Before the integration of dedicated 3D canvas modules, many computer vision teams had to rely on fragmented tools for 2D and 3D annotation. CVAT-CANVAS-3D fills this gap by unifying the annotation experience. For developers, this means a single platform where they can manage both image-based and point-cloud-based labels, reducing the overhead of switching between different software environments.

The rise of autonomous driving and robotics has created a massive demand for 3D bounding box (3D Bbox) annotation. Because 3D data is inherently more complex than 2D images, the precision of the canvas is critical. CVAT-CANVAS-3D provides the necessary spatial reference and orientation tools to ensure that cuboids are aligned correctly within the 3D environment, which is essential for training models that must interact with the real world.

By being open-source and MIT-licensed, it allows research teams and enterprises to self-host their annotation infrastructure, ensuring that sensitive 3D data—often containing high-resolution LiDAR scans of private environments—never leaves their own secure servers.

Key Features

• 3D Cuboid Annotation: Create and manage 3D bounding boxes for objects like vehicles and buildings. This allows for the capture of depth, orientation, and size in a 3D space.
• Multi-View Perspective: The canvas provides multiple views (perspective, top, side, and front) to ensure precise alignment of 3D objects from different angles.
• Interactive Drawing Tools: A user-friendly interface for drawing and editing 3D shapes directly on the canvas, reducing the manual effort required for labeling.
• Interpolation and Tracking: Support for tracking 3D objects across frames using interpolation to predict the position of objects in subsequent frames, which is critical for video-based 3D data.
• Orientation Reference: Visual indicators (X-red, Y-green, Z-blue) for cuboid axis orientation, providing a visual reference for the cuboid’s alignment within the 3D environment.
• TypeScript API: A robust API that allows developers to programmatically control the canvas, setup frame data, and render 3D annotations.
• Performance Optimization: Built for efficiency to handle dense point clouds and complex 3D scenes without lagging the browser interface.
• Contextual Image Integration: The ability to include contextual images within the 3D canvas to provide additional visual cues for the annotator.

How CVAT-CANVAS-3D Compares

When comparing CVAT-CANVAS-3D to other 3D annotation tools, the primary differentiator is its open-source nature and its integration into a comprehensive 2D/3D ecosystem. While tools like BasicAI offer more advanced point intensity visualization—which is critical for distinguishing reflective objects in LiDAR data—CVAT-CANVAS-3D is often the preferred choice for teams that require absolute data privacy and full control over their infrastructure.

For academic research or small-to-medium projects, CVAT-CANVAS-3D provides a sufficient and highly flexible toolset. However, for massive enterprise-scale LiDAR projects that require advanced AI-assisted automation and high-end rendering engines for extremely dense point clouds, proprietary platforms like Encord may offer a more streamlined, albeit more expensive, experience.

Getting Started: Installation

CVAT-CANVAS-3D is a module within the larger CVAT repository. To use it, you must first set up the CVAT environment. Prerequisites include Docker Engine, Docker Compose, and Git.

Using Docker (Standard Installation)

The most common way to deploy CVAT is via Docker Compose, which handles all dependencies including the 3D canvas module.

git clone https://github.com/openvinotoolkit/cvat
cd cvat
docker compose up -d

Building the Module from Source

If you are developing for the module or need to customize the 3D canvas, you can build it independently using Yarn.

# Navigate to the canvas-3d directory
cd cvat-canvas3d

# Build for production
yarn run build

# Build for development (without minification)
yarn run build --mode=development

After building, the output is placed in the dist directory, which can then be integrated into your web application.

How to Use CVAT-CANVAS-3D

The 3D canvas is used primarily through the CVAT UI, but it can also be implemented programmatically. The basic workflow involves initializing the canvas, setting up the frame data (the point cloud), and then rendering the annotations.

The user typically starts by creating a 3D task in CVAT, uploading a .zip archive containing point cloud data (such as .pcd or .bin files). Once the task is opened, the 3D canvas renders the point cloud and provides the tools to draw cuboids.

If you are using the module as a library, you initialize a Canvas3d instance and append the resulting HTML views to your container. This allows you to create a custom 3D annotation interface based on the CVAT engine.

Code Examples

The following examples demonstrate how to interact with the Canvas3d class in TypeScript/JavaScript.

Initializing the Canvas

This snippet shows how to create an instance of the 3D canvas and append its multiple views to an HTML container.

// Create an instance of a canvas
const canvas = new window.canvas.Canvas3d();

console.log('Version ', window.canvas.CanvasVersion);
console.log('Current mode is ', window.canvas.mode());

// Put canvas to a html container
const views = canvas.html();
htmlContainer.appendChild(views.perspective);
htmlContainer.appendChild(views.top);
htmlContainer.appendChild(views.side);
htmlContainer.appendChild(views.front);

Managing 3D Annotations

The Canvas3d interface defines several methods for controlling the 3D environment and the annotation state.

// Example of setting up frame data and object states
canvas.setup(frameData, objectStates);

// Render the current state of the canvas
canvas.render();

// Fit the canvas to the current data
canvas.fitCanvas();

// Activate a specific object for editing
canvas.activate(clientID, attributeID);

Real-World Use Cases

CVAT-CANVAS-3D is essential for any project that requires the translation of raw 3D spatial data into labeled ground truth for AI models.

• Autonomous Driving: Labeling LiDAR point clouds to detect vehicles, pedestrians, and road boundaries. This is critical for training perception systems that must understand depth and orientation.
• Robotics: Annotating 3D scenes for robotic arm manipulation or warehouse automation, where the robot must accurately identify the 3D position and size of objects.
• 
  Industrial Inspection: Using 3D scans of infrastructure (like bridges or pipelines) to label defects or structural anomalies in a 3D volumetric space.
• Urban Planning: Labeling large-scale city scans to automatically categorize buildings, trees, and street furniture for digital twin creation.

Contributing to CVAT-CANVAS-3D

As an open-source project, CVAT encourages contributions from the community. Because the 3D canvas is a core part of the CVAT repository, contributions should follow the general CVAT contribution guidelines.

Developers can report bugs via the GitHub Issue Tracker, request new features, and submit pull requests. It is recommended to check if an issue is already open before creating a new one. For those looking to get started, the project maintains a list of “good first issues” to help new contributors integrate into the project.

The project adheres to a strict code of conduct to ensure a constructive and respectful environment for all developers.

Community and Support

CVAT is supported by a large and active community of developers and AI practitioners. Support can be found through several official channels:

• GitHub Discussions: The primary hub for community questions, feature requests, and technical troubleshooting.
• Discord: For real-time communication and quick support from other users and the community.
• Official Documentation: A comprehensive guide to using CVAT and its 3D annotation tools.
• CVAT Academy: For structured learning materials and tutorials on data annotation best practices.

Conclusion

CVAT-CANVAS-3D provides a critical bridge between raw 3D data and the model-ready datasets required for modern computer vision. By offering a high-performance, open-source canvas for 3D cuboid and shape annotation, it allows teams to avoid the expensive lock-in of proprietary tools while maintaining full control over their data infrastructure.

While it may lack some of the most advanced point intensity visualization features found in specialized LiDAR tools, its integration into the broader CVAT ecosystem makes it a powerful and flexible choice for most 3D labeling tasks.

Star the repo, try the quickstart, and join the community to start building high-quality 3D datasets.

Resources

• CVAT GitHub Repository
• CVAT Official Documentation
• CVAT Academy