Introduction

The rise of Large Language Models (LLMs) has created a new paradigm in software development, yet for many, their inner workings remain a mystery, hidden behind APIs. For developers who want to move beyond being a consumer and truly understand the fundamentals, a new wave of educational projects is emerging. One of the most notable for the Rust community is Inference School, an open-source, hands-on course that guides you through implementing an LLM from scratch. It’s a comprehensive curriculum for anyone looking to demystify the magic behind models like GPT.

What Is Inference School?

Inference School is a self-contained educational project on GitHub that provides lessons and exercises for learning how to implement Large Language Models from the ground up using Rust. Created by Alvaro Videla and licensed under the MIT license, the project is heavily inspired by Andrej Karpathy’s influential `makemore` and `llm.c` projects. Its goal is not to create a production-ready inference engine, but to provide a clear, step-by-step, code-first educational experience for developers seeking a deep, foundational understanding of how transformer-based models work.

The entire curriculum is structured as a series of lessons within a single repository. Each lesson is a self-contained Rust program that builds upon the concepts of the previous one. This structured approach takes you from the basics of data loading and tensor manipulation all the way to implementing the core self-attention mechanism that powers modern LLMs.

Why Inference School Matters

While Python is the undisputed lingua franca of machine learning research, Rust is rapidly gaining traction in the AI/ML space for its performance, memory safety, and robust tooling. Inference School fills a critical gap by providing a high-quality educational resource specifically for the Rust ecosystem. Before this, aspiring Rust developers had to mentally translate concepts from Python or C, a process that can obscure the unique challenges and advantages of implementing these systems in Rust.

The project’s philosophy of building everything from first principles is its most important attribute. In a world of high-level libraries like PyTorch and TensorFlow, it’s easy to build complex models without ever understanding how they actually work. By forcing you to implement your own Tensor library and automatic differentiation engine, Inference School strips away the abstractions and reveals the core mechanics. This foundational knowledge is invaluable for anyone who wants to contribute to ML frameworks, conduct novel research, or build highly optimized, custom AI solutions.

Key Features: The Curriculum

The core of Inference School is its structured curriculum, where each lesson is a feature designed to build your understanding incrementally.

• Guided Lesson Structure: The project is organized into a clear sequence of lessons, starting from a simple ‘hello world’ to establish the project structure and progressing methodically through more complex topics. This ensures you are never overwhelmed and can learn at your own pace.
• Data Loading and Tokenization: Early lessons cover the crucial first steps of any ML project: how to load a dataset from disk and prepare it for the model by converting text into numerical tokens.
• Custom Tensor Library: Instead of importing a pre-built library, you will implement your own multi-dimensional array (Tensor) library. This exercise provides a deep appreciation for the fundamental data structure of all modern deep learning.
• Autodiff From First Principles: One of the most magical parts of deep learning is how models learn via backpropagation. Inference School demystifies this by guiding you through the implementation of an automatic differentiation engine.
• Building a Neural Network: With the core components in place, you will construct and train a Multi-Layer Perceptron (MLP), learning the mechanics of a training loop, loss calculation, and weight updates.
• Self-Attention Implementation: The curriculum culminates in what is arguably the most important lesson: building the self-attention mechanism. This is the core innovation of the Transformer architecture and understanding its implementation is key to understanding all modern LLMs.

How Inference School Compares

Inference School is best understood by comparing it to other resources aimed at teaching LLM fundamentals. Its primary distinction is the choice of programming language and its self-contained, project-based format.

Inference School vs. Karpathy’s `makemore`/`llm.c`: Karpathy’s work is the direct inspiration, and the educational goals are very similar. The main difference is the medium. Karpathy’s content is primarily video-based, where he live-codes the solution, which is excellent for building intuition. Inference School is a pure code artifact; you learn by reading, running, and modifying the Rust code directly. It is ideal for learners who prefer a project-based, self-guided approach and specifically want to do so in Rust.

Inference School vs. Fast.ai: Fast.ai represents a completely different philosophy. It uses a top-down approach, teaching you to train state-of-the-art models on real-world problems from day one, and only later dives into the underlying theory. Inference School is the opposite. It is entirely bottom-up, ensuring you understand every single line of code and every mathematical operation before you even get to a complete model. Fast.ai is for creating practitioners quickly; Inference School is for creating deep understanding in developers.

Getting Started: Installation

Getting started with Inference School is simple, as it has no external dependencies beyond the standard Rust toolchain.

Prerequisites

You must have the Rust programming language and its package manager, Cargo, installed on your system. You can install them together by following the official instructions at rustup.rs.

Running the Lessons

First, clone the repository from GitHub to your local machine.

git clone https://github.com/videlalvaro/inference-school.git
cd inference-school

Each lesson is a separate binary within the Cargo workspace. To run a specific lesson, for example `lesson-0-hello-world`, you use the `cargo run` command with the `–bin` flag.

cargo run --bin lesson-0-hello-world

This command will compile and execute the code for the specified lesson, printing its output to the console.

How to Use Inference School

The intended workflow is to progress through the lessons sequentially, as each one builds on concepts introduced previously. For each lesson, you should start by reading its corresponding README file (e.g., `lessons/lesson-2-tensors/README.md`). These files often contain explanations of the theoretical concepts being introduced.

After reading the theory, dive into the source code (`main.rs` in each lesson’s directory). Read through the implementation, paying close attention to the comments and the overall structure. Finally, run the code as described in the installation section to see it in action. The best way to learn is to experiment: try changing some of the parameters, printing out intermediate values, and even attempting to add small features to solidify your understanding.

Code Examples

The entire repository is a collection of code examples. Here is a small, illustrative snippet from `lesson-2-tensors`, where a basic `Tensor` struct is first introduced. This highlights the project’s from-scratch nature.

Creating a Tensor

This code defines a simple Tensor struct that holds a flat vector of floating-point numbers and a `shape` vector to describe its dimensions. It also includes a constructor to create a new tensor filled with zeros.

#[derive(Debug, Clone)]
pub struct Tensor {
    pub data: Vec<f32>,
    pub shape: Vec<usize>,
}

impl Tensor {
    pub fn new(shape: &[usize]) -> Self {
        let size = shape.iter().product();
        Self {
            data: vec![0.0; size],
            shape: shape.to_vec(),
        }
    }
}

// Usage in the main function of the lesson
let t = Tensor::new(&[2, 3]);
println!("{:?}", t);
// Output will show a Tensor with shape [2, 3] and data [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

This simple example demonstrates the foundational, hands-on approach of the course. You are not just using a tensor; you are building the very definition of one.

Real-World Use Cases (Learning Outcomes)

While Inference School is not a production library, completing it equips you with skills for several real-world scenarios:

• Deconstruct Research Papers: After implementing a transformer from scratch, you will have the foundational knowledge to read modern AI research papers and understand them at an implementation level.
• Contribute to ML Frameworks: You will gain the low-level understanding necessary to contribute to open-source ML libraries in the Rust ecosystem like Candle, Burn, or dfdx.
• Build High-Performance Inference Engines: The course provides the necessary background to start building custom, highly optimized inference solutions for specific hardware or use cases where generic libraries are too slow or bloated.
• Pass Advanced AI Job Interviews: Completing this course will prepare you to answer deep, systems-level questions about how neural networks work, setting you apart from candidates who only have high-level library knowledge.
• Learn Rust in a Meaningful Context: For developers new to Rust, this project serves as an excellent, non-trivial application to learn the language, its ownership model, and its ecosystem in the context of scientific computing.

Contributing to Inference School

As an open-source educational project, contributions from the community are valuable. There is no formal `CONTRIBUTING.md` file, so the best approach is to follow standard GitHub etiquette. You can help by improving documentation, clarifying comments in the code, fixing bugs, or even proposing new lessons or exercises.

Before undertaking significant work, it is recommended to open an issue on the GitHub repository to discuss your proposed changes with the author. This ensures your contribution aligns with the project’s goals. Submitting a pull request with clear explanations of your changes is the final step.

Community and Support

The primary place for all community interaction and support is the GitHub repository itself. All communication is managed through GitHub’s built-in features.

• GitHub Issues: Use the issues tab to ask questions, report problems you encounter in the lessons, or suggest improvements. This is the main channel for support.

There are currently no dedicated Discord, Slack, or other community forums. The project is best suited for self-motivated learners who are comfortable working within the GitHub ecosystem.

Conclusion

Inference School is a remarkable achievement in open-source education. It successfully demystifies the complex world of Large Language Models by breaking them down into understandable, implementable parts. It is not a shortcut or a high-level framework; it is a deep dive into the fundamental principles, and its value lies in that depth. For any developer, especially within the Rust community, who has ever wondered “how does ChatGPT *really* work?”, this project is the definitive hands-on answer.

If you are looking to build a production application, you should reach for a mature library. But if your goal is to acquire a robust, lasting understanding of the technology that will define the next decade of software, there are few better ways to invest your time. The best way to start is to clone the repository, fire up your terminal, and run the first lesson.

Resources

• Official Inference School GitHub Repository: The source code, lessons, and README files.
• Andrej Karpathy’s makemore Repository: The inspirational project for Inference School, implemented in Python/Jupyter.
• The Rust Programming Language: The official website for the Rust language, where you can find installation instructions and documentation.