Introduction

Have you ever wondered if you could create an AI that texts like you? With the rise of large language models, the idea of training a model on personal data is fascinating, yet often seems complex and raises significant privacy concerns. Enter texts-to-transformer, a remarkable open-source project by Doriandarko that provides a complete, privacy-conscious pipeline to train a small transformer model from scratch using your iMessage history, all locally on your Apple Silicon Mac. With over 400 GitHub stars, it demystifies model training and puts personal AI into your hands.

What Is texts-to-transformer?

Texts-to-transformer is a comprehensive Python-based toolkit that allows you to train a custom, decoder-only transformer model on your iMessage history. It is designed to run entirely on a local Apple Silicon Mac, leveraging Apple’s MLX framework for efficient on-device training. The project is not just a training script; it’s an end-to-end pipeline that covers data snapshotting, text extraction, pseudonymization, dataset splitting, tokenizer training, model training, evaluation, and even a local chat interface for inference. Everything—the model and the tokenizer—is trained from zero, without relying on any pretrained components, offering a true from-scratch experience.

The entire project is released under the permissive MIT license, making it completely free to use and modify. It is explicitly designed as a tool for creating a small personal style model, not a general-purpose assistant. Its primary goal is to learn your unique phrasing, rhythm, and slang, providing a fascinating look into your own communication patterns.

Why texts-to-transformer Matters

This project is significant for several reasons. First, it addresses the complex and often intimidating process of training a language model and makes it accessible to a wider audience of developers and hobbyists. Instead of requiring deep knowledge of complex frameworks or cloud infrastructure, it provides a series of simple, well-documented command-line tools that guide the user through every step, from data extraction to model interaction. This lowers the barrier to entry for hands-on NLP experimentation.

Second, and perhaps most importantly, texts-to-transformer places an extraordinary emphasis on safety and privacy. In an era where personal data is a sensitive commodity, the project’s design is a case study in responsible development. It never touches the live iMessage database, works on a read-only backup, redacts PII, and pseudonymizes identities before any data is saved. By keeping the entire process local to the user’s machine, it ensures that your private conversations are never exposed to the cloud or third parties. This privacy-first approach is a crucial counterpoint to the trend of relying on centralized, proprietary model providers.

Key Features

Texts-to-transformer is a feature-complete pipeline. Here are some of its core components:

• Safe Data Pipeline: The project includes scripts to safely create a read-only SQLite backup of your Messages database, ensuring your live data is never at risk. All processing happens from this private, local snapshot.
• Privacy-First Processing: A critical feature is the robust text extraction and pseudonymization process. It redacts URLs, emails, and phone numbers, and replaces chat identifiers with keyed HMAC pseudonyms to protect your identity and the identities of your contacts.
• Leakage-Resistant Dataset Splits: It intelligently splits your conversation data into chronological train, validation, and test sets with guard bands between them. This prevents data leakage and ensures the model is evaluated on genuinely unseen data.
• Custom Tokenizer Training: Instead of using a generic tokenizer, you train your own byte-level Byte-Pair Encoding (BPE) tokenizer from your text. This allows the model to perfectly capture your unique slang, emoji usage, and even misspellings.
• Custom Transformer Architecture: The project provides its own efficient, decoder-only Transformer implementation built with MLX. It comes with two presets: a 1.38M-parameter model and a larger 6.16M-parameter version for those with extensive message histories.
• End-to-End MLX Training: It features a complete training loop using Apple’s MLX framework, including AdamW optimization, learning rate scheduling, gradient clipping, and resumable checkpoints.
• Thorough Evaluation and Auditing: After training, you can evaluate the model’s perplexity on the test set and run checks for n-gram overlap to gauge potential memorization of sensitive training data.
• Local Chat Interface: Once the model is exported, you can interact with it directly in your terminal. The interface allows you to type a message and see how your personalized model would suggest you reply, all without ever sending a real message.

How texts-to-transformer Compares

This project occupies a unique niche. It’s not trying to compete with large, general-purpose training frameworks but rather to provide a specialized, simplified experience.

The key differentiator for texts-to-transformer is its holistic, privacy-centric, and highly opinionated approach. While a library like Hugging Face Transformers is infinitely more powerful and flexible, it requires the user to write the boilerplate code for data loading, processing, and training. Texts-to-transformer provides all of that boilerplate for one specific, compelling use case. Compared to fine-tuning a model via a cloud API (like OpenAI’s), this project offers complete data privacy and zero cost, at the trade-off of producing a much smaller, less capable model.

It is the perfect tool for someone whose goal is to learn how transformers work by building one from the ground up on a dataset they know intimately, without the cost and privacy risks associated with other methods.

Getting Started: Installation

The project is designed for Apple Silicon Macs and uses modern Python tooling.

Prerequisites

• An Apple Silicon Mac (M1 or newer) with macOS 14+
• At least 16 GB of RAM is recommended.
• Homebrew for installing the `uv` package manager.
• Full Disk Access granted to your Terminal application to allow the initial database snapshot.

Installation Steps

The process is managed through a series of clear command-line steps.

# 1. Clone the repository
git clone https://github.com/Doriandarko/texts-to-transformer.git
cd texts-to-transformer

# 2. Install the `uv` Python package manager (if you don't have it)
brew install uv

# 3. Create the virtual environment and install dependencies
uv sync

# 4. Run the doctor command to verify your setup
uv run imessage-mlx doctor

The `doctor` command is a crucial first step that checks for Apple Silicon, MLX support, disk space, and most importantly, whether it has the necessary read-only access to the Messages database. If it fails the access check, you must grant Full Disk Access in your Mac’s System Settings and restart your terminal.

How to Use texts-to-transformer

The entire process, from data to model, is broken down into a series of `uv run imessage-mlx` commands. You must run them in order, as each step produces artifacts required by the next.

First, you create a safe, read-only snapshot of your iMessage database. Then, you run the `prepare` command to extract, clean, and pseudonymize the text data, splitting it into training, validation, and test sets. With your private dataset ready, you train a new tokenizer on your training data, which creates a vocabulary tailored to your speaking style. Finally, you run the `train` command, pointing to your data and tokenizer, to begin training the transformer model from random initialization. After training and evaluation, you can use the `chat` command to interact with your creation.

Code Examples

All interactions with the project are through the command line. The following commands are a condensed version of the full workflow described in the README.

Example 1: Create a Safe Data Snapshot

This command creates a read-only copy of your `chat.db` in the `work/` directory, which is ignored by Git. All subsequent steps will use this safe copy.

uv run imessage-mlx snapshot --config configs/data.yaml

Example 2: Prepare the Dataset

This command runs the full extraction, cleaning, pseudonymization, and splitting pipeline. It is the most critical data processing step.

uv run imessage-mlx prepare --config configs/data.yaml

Example 3: Train the Tokenizer

Using only the training split, this creates a custom 4,096-token BPE tokenizer that understands your specific vocabulary and style.

uv run imessage-mlx train-tokenizer 
 --train work/splits/train.jsonl 
 --output outputs/tokenizer 
 --vocab-size 4096

Example 4: Train the Model

This kicks off the main training process using the default 1.38M parameter model configuration. It saves checkpoints and logs to the `outputs/runs/my-model` directory.

uv run imessage-mlx train 
 --config configs/model-1m.yaml 
 --data work/tokens 
 --tokenizer outputs/tokenizer 
 --output outputs/runs/my-model

Example 5: Chat with Your Model

After exporting the final model, you can use this command to interact with it in your terminal.

uv run imessage-mlx chat --model outputs/final

Real-World Use Cases

• Digital Self-Reflection: Analyze your own communication patterns, slang, and common phrases by seeing what a model trained on your data produces.
• Creative Writing Assistant: Generate text that mimics your personal style to help overcome writer’s block or explore creative variations of your own voice.
• Educational Tool: For students and developers new to NLP, this project provides a tangible, end-to-end example of how a transformer is built and trained from the ground up on a unique dataset.
• Personalized Chatbot: While not a capable assistant, the model can be used to power a simple chatbot that replies with your specific mannerisms, useful for automated but personalized responses.
• Data Privacy Demonstration: The codebase serves as an excellent reference for how to build privacy-preserving machine learning pipelines on sensitive local data.

Contributing to texts-to-transformer

The project is hosted on GitHub and contributions are managed through the standard workflow. While it doesn’t have a formal `CONTRIBUTING.md` file, you can participate by opening a GitHub Issue to report bugs, suggest features, or ask questions. If you want to contribute code, it is best practice to fork the repository, create a new branch for your feature, and then submit a Pull Request for the maintainer to review.

Community and Support

The primary channel for support and community discussion is the GitHub Issues page. This is the best place to post troubleshooting questions or discuss potential improvements with the creator. The repository also includes a detailed troubleshooting guide in its documentation to help with common issues like Full Disk Access failures or insufficient data problems.

Conclusion

Texts-to-transformer is an exemplary open-source project that successfully balances accessibility, education, and an uncompromising commitment to user privacy. It takes the abstract, complex world of transformer training and grounds it in a personal, tangible, and fun use case. By providing a complete, step-by-step pipeline that runs entirely on a local machine, it empowers users to not only create a personalized AI but also to understand the entire lifecycle of a machine learning model.

This is not a tool for building the next ChatGPT. Instead, it offers something arguably more valuable: a safe and insightful way to explore the intersection of language, identity, and artificial intelligence. If you are an Apple Silicon Mac user with an interest in NLP, this project is one of the most rewarding and well-executed experiences available. It’s a fantastic weekend project that will leave you with a deeper appreciation for both your own linguistic quirks and the magic of modern machine learning.

We highly recommend you check out the repository, give it a star on GitHub, and take the journey of training a transformer on your own texts.

Resources

• Official texts-to-transformer GitHub Repository
• Apple’s MLX Framework: The underlying machine learning library that powers the training.
• Project Privacy Documentation: A must-read before starting the data pipeline.