Introduction

Extracting structured data from complex documents like legal contracts, financial reports, or technical manuals often requires writing endless boilerplate code and complex prompting strategies. ContextGem is an open-source LLM framework that simplifies this process by providing powerful abstractions that handle the heavy lifting of data extraction. With its ability to map extractions back to source references and automate data modeling, ContextGem allows developers to move from a raw document to a structured JSON output with minimal effort.

What Is ContextGem?

ContextGem is a free, open-source LLM framework that makes it radically easier to extract structured data and insights from documents—with minimal code. It is written in Python and licensed under the Apache 2.0 license, allowing for wide enterprise adoption. The framework is designed to strike a balance between ease of use, customizability, and accuracy, specifically targeting the gap between simple prompting and full-scale RAG (Retrieval-Augmented Generation) systems.

Unlike traditional extraction tools, ContextGem doesn’t just return a string; it provides a unified, serializable document storage model that includes precise paragraph- and sentence-level references, automatic justifications for every extracted item, and support for hierarchical multi-aspect extraction.

Why ContextGem Matters

For most developers, the most time-consuming part of building an LLM-based extraction pipeline is not the model itself, but the “plumbing”—the prompt engineering, the output validation, and the mapping of results back to the original source for verification. ContextGem eliminates this boilerplate by providing a declarative syntax where you describe what to extract, and the framework handles how it is done.

This approach significantly reduces development time—often by 3-5x—and increases the reliability of the output. By leveraging the long context windows of modern LLMs, ContextGem avoids the pitfalls of traditional RAG, which can often miss subtle nuances or inconsistencies across a large document. Instead, it analyzes the entire document context to ensure high extraction accuracy.

As enterprises move toward agentic document intelligence, tools like ContextGem provide the necessary foundation for turning unstructured text into high-fidelity structured data that can be fed into deterministic tools and billing calculations, as seen in real-world applications for scaffolding project files and invoice generation.

Key Features

• Automated Dynamic Prompts: The framework auto-generates prompts based on your natural-language extraction targets, removing the need to manually craft and maintain complex prompt templates.
• Automated Data Modeling: ContextGem provides built-in validators and data models that ensure the LLM output conforms to the requested structure, reducing the need for manual parsing and post-processing.
• Granular Reference Mapping: Every extracted piece of data is mapped back to its precise location in the source document at the paragraph and sentence level, allowing for easy verification and auditing.
• Built-in Justifications: The framework requires the LLM to provide reasoning (justifications) for each extraction, which increases transparency and reduces hallucinations.
• Nested Context Extraction: It supports hierarchical extraction, where you can define aspects (broad topics) and concepts (specific data points) within those aspects, mirroring the structure of complex documents.
• Unified Declarative Pipeline: The entire extraction workflow is a single, reusable, and serializable pipeline, making it easy to deploy and version control.
• Neural Segmentation (SaT): Using wtpsplit SaT models, ContextGem ensures accurate text segmentation into paragraphs and sentences, which is critical for the granular reference mapping feature.
• Multilingual Support: The framework supports I/O without requiring explicit prompting for language, allowing it to work across different languages seamlessly.
• LiteLLM Integration: Through LiteLLM, ContextGem supports a vast array of cloud LLMs (OpenAI, Anthropic, Google, Azure) and local models (Ollama, LM Studio), providing a unified interface for easy provider switching.
• Native DOCX Converter: The project includes a native converter to handle Word documents, which are the primary format for most business and legal documents.

How ContextGem Compares

ContextGem differs from general-purpose LLM frameworks like LlamaIndex by focusing specifically on the extraction of structured data rather than the retrieval of information. While LlamaIndex is excellent for building RAG systems where you query a knowledge base, ContextGem is designed for scenarios where you need to turn a specific document into a structured record (e.g., turning a lease agreement into a database entry).

Compared to manual prompting, ContextGem provides a standardized way to handle validation and reference mapping. In a custom setup, a developer would have to manually implement the logic to find where a specific piece of information was found in the original text—a process that is notoriously difficult to get right. ContextGem handles this automatically, making it the superior choice for high-stakes documents where auditability is required.

Getting Started: Installation

ContextGem can be installed via the standard Python package manager. The project recommends using uv for faster dependency management.

Using uv (Recommended)

uv add contextgem

Using pip

pip install -U contextgem

Prerequisites: You will need Python 3.10 or higher and an API key for your chosen LLM provider (e.g., OpenAI, Anthropic, or a local provider via Ollama).

How to Use ContextGem

The basic workflow in ContextGem involves creating a Document object, defining the Aspects or Concepts you want to extract, and then using a DocumentLLM to perform the extraction.

First, you load your raw text into a Document. Then, you define an Aspect—a broad category of information. For example, if you are analyzing a contract, you might define an aspect for “Intellectual Property Rights.” You then provide a natural language description of what that aspect covers.

Finally, you initialize a DocumentLLM with your model of choice and call the extract_all method. The framework handles the prompt construction, the LLM call, and the parsing of the results back into the document object, which now contains the extracted items with their source references.

Code Examples

Below are examples of how to implement extraction using ContextGem, ranging from simple aspect extraction to more complex concept extraction.

Simple Aspect Extraction

This example shows how to extract broad topics from a document using a simple description.

from contextgem import Aspect, Document, DocumentLLM

# Define the document
doc = Document(raw_text="Your document text here...")

# Define what to extract
doc.aspects = [
    Aspect(name="Intellectual property", description="Clauses on intellectual property rights")
]

# Extract with any LLM
llm = DocumentLLM(model="openai/gpt-4o", api_key="your_api_key")
doc = llm.extract_all(doc)

# Get results
print(doc.aspects[0].extracted_items)

Complex Concept Extraction

This example demonstrates extracting specific, typed data points (Concepts) from a document, such as dates or strings, with precise references.

from contextgem import Document, DocumentLLM, StringConcept

# Load document
doc = Document(raw_text="The contract was signed on 2023-10-12 by Acme Corp.")

# Define specific concepts to extract
doc.concepts = [
    StringConcept(name="Company Name", description="The name of the legal entity signing the contract")
]

# Perform extraction
llm = DocumentLLM(model="anthropic/claude-3-5-sonnet", api_key="your_api_key")
doc = llm.extract_all(doc)

# The result includes the text, the reference to the paragraph/sentence, and the justification
print(doc.concepts[0].extracted_items[0].text)
print(doc.concepts[0].extracted_items[0].reference)

Real-World Use Cases

ContextGem is particularly effective in industries where document accuracy and auditability are non-negotiable.

• Legal Tech: Lawyers can use ContextGem to analyze thousands of pages of contracts to identify specific clauses (e.g., “Change of Control” or “Non-Compete”) and map them directly to the page and paragraph for quick verification.
• Financial Analysis: Analysts can extract key financial metrics from quarterly reports, ensuring that every number extracted is backed by a justification and a direct reference to the source text.
• Compliance and Audit: Compliance officers can automate the scanning of internal policies to ensure they align with new regulations, identifying gaps or anomalies in the text.
• Enterprise Document Intelligence: Companies can build multi-agent systems that process large volumes of project files to perform complex billing calculations based on extracted structured data.

Contributing to ContextGem

The maintainers of ContextGem welcome contributions from the community. Whether you are fixing a typo in the documentation or developing a new feature, you can get started by reviewing the project’s Contributor Guidelines on GitHub.

The project follows a standard GitHub flow: fork the repository, create a feature branch, and submit a pull request. All contributions are submitted under the Apache 2.0 license. The project also maintains a strict security policy, with automated scanning using CodeQL, Bandit, and Snyk to ensure the project remains secure for enterprise use.

Community and Support

ContextGem is developed by Shcherbak AI, an enterprise AI engineering company based in Oslo, Norway. It is developed as an open-source project to provide a baseline for AI developers building document intelligence platforms.

The primary channel for support and feedback is the //issues section of the GitHub repository. For more detailed technical guidance, users can visit the official documentation site at contextgem.dev. The official source of truth for documentation is the website, as the maintainers warn against unauthorized mirrors.

Conclusion

ContextGem is the right choice for developers who need to turn unstructured documents into structured data with high precision and minimal boilerplate. It is a powerful alternative to the general-purpose RAG frameworks when the goal is a complete, structured extraction of a document’s contents rather than a simple query-response interaction.

While the project is newer than some of the larger AI frameworks, its specific focus on the “plumbing” of extraction—reference mapping, justifications, and data modeling—makes it it a highly specialized tool that fills a critical gap in the AI ecosystem. If you are working with legal, financial, or business documents, ContextGem is a highly recommended starting point.

Star the repo, try the quickstart, and join the community to help shape the future of document intelligence.

Resources

• ContextGem GitHub Repository
• Official ContextGem Documentation