Introduction

Finding accurate, comprehensive answers to complex questions often requires more than a simple search query; it requires a systematic process of browsing, reading, and synthesizing information. II-Researcher is an open-source deep search agent that automates this entire research workflow, allowing users to generate detailed reports with verifiable references. With its ability to autonomously navigate the web and reflect on its findings, it replaces the manual drudgery of tab-hunting and fragmented note-taking for developers and researchers.

What Is II-Researcher?

II-Researcher is a deep search agent framework that autonomously searches the web, visits and reads pages, reflects on intermediate findings, and synthesizes a comprehensive final answer with references. Developed by the Intelligent Internet team, it is designed to mimic the workflow of a human researcher—planning a strategy, gathering evidence, and self-correcting until a complete answer is reached.

Built primarily in Python, the project is released under the Apache License 2.0, ensuring it remains open and extensible. It provides a complete, self-hostable stack including a Python library, a CLI, a streaming FastAPI backend, and a Next.js web UI, making it accessible to both command-line power users and those who prefer a graphical interface.

Why II-Researcher Matters

In an era of AI-generated “slop” and hallucinated citations, the need for grounded, verifiable research is paramount. Most standard LLM chatbots provide immediate answers based on internal training data, which can be outdated or inaccurate. II-Researcher fills this gap by prioritizing external grounding through real-time web access and a multi-step reasoning loop.

By open-sourcing this framework, Intelligent Internet counters the trend of proprietary “Deep Research” tools that are often black boxes. It empowers developers to bring their own API keys (BYOK) and choose their own LLM providers, providing full control over the cost, privacy, and the specific models used for reasoning and synthesis. This transparency is critical for academic and professional research where the provenance of information is as important as the answer itself.

Key Features

• Intelligent Web Search: Utilizes pluggable search providers such as Tavily, SerpAPI, and Jina to ensure high-quality, agent-optimized search results.
• Advanced Web Scraping: Supports multiple content extraction providers including Firecrawl, Browser, BeautifulSoup, and Tavily Extract, with specialized handling for PDFs and YouTube content.
• Multi-Step Reasoning and Reflection: The agent does not simply summarize; it thinks, acts, and self-corrects. It reflects on knowledge gaps and triggers follow-up searches to fill those gaps before finalizing the report.
• Context Compression: Employs embedding-based and LLM-based compression to efficiently consolidate essential facts from massive amounts of retrieved web content without hitting token limits.
• Streaming FastAPI Backend: Provides a real-time stream of the agent’s reasoning process and answer generation, allowing users to monitor the research trajectory in real-time.
• MCP Server Integration: Includes a Model Context Protocol (MCP) server that allows II-Researcher to be plugged directly into Claude Desktop and other MCP-compatible clients as a research tool.
• Configurable LLM Pipelines: Integrates with LiteLLM to support a wide array of AI models (e.g., GPT-4o, DeepSeek), allowing users to tune the reasoning engine to their specific needs.
• Comprehensive Answer Generation: Produces structured reports with inline citations and a full reference list, ensuring every claim is backed by a source.

How II-Researcher Compares

II-Researcher competes in the space of autonomous research agents, facing off against proprietary tools like OpenAI’s Deep Research and other open-source alternatives like GPT-Researcher.

The primary differentiator for II-Researcher is its focus on the full stack. While many research agents are just Python scripts, II-Researcher provides a production-ready backend and a polished Next.js frontend, making it a complete application rather than just a library. Its integration with the Model Context Protocol (MCP) also makes it a powerful extension for existing AI clients like Claude, allowing users to bring deep research capabilities into their primary chat interface.

Compared to GPT-Researcher, II-Researcher emphasizes a more explicit reflection cycle where the agent critically evaluates its own findings to identify gaps before the final synthesis. This reduces the likelihood of superficial summaries and increases the depth of the final report.

Getting Started: Installation

II-Researcher can be installed via several methods depending on your needs. Ensure you have Python 3.7+ installed on your system.

Installation from Source

Clone the repository and install the package in editable mode:

git clone https://github.com/Intelligent-Internet/ii-researcher.git
cd ii-researcher
pip install -e .

Docker Deployment (Easy Mode)

For those who prefer containerization, II-Researcher provides helper scripts to spin up the entire stack (backend and frontend) quickly:

chmod +x start.sh stop.sh
./start.sh

Once the containers are running, the web interface is typically available at http://localhost:3000.

Prerequisites: API Keys

Because II-Researcher is a BYOK (Bring Your Own Key) tool, you must configure your environment variables. Create a .env file or export them in your shell:

export OPENAI_API_KEY="your-openai-api-key"
export TAVILY_API_KEY="your-tavily-api-key"
export SERPAPI_API_KEY="your-serpapi-api-key"
export FIRECRAWL_API_KEY="your-firecrawl-api-key"

How to Use II-Researcher

The simplest way to start is using the Command Line Interface (CLI). Once your API keys are configured, you can run a deep search task directly from your terminal.

Run the following command to research a specific topic:

python ii_researcher/cli.py --question "What are the latest trends in autonomous AI agents for 2026?" --stream

The --stream flag is highly recommended as it allows you to see the agent’s internal reasoning process—what it’s searching for, which pages it’s reading, and how it’s reflecting on the information—in real-time. This transparency is a key part of the II-Researcher experience.

If you prefer a GUI, start the backend API server with python api.py and then run the Next.js frontend. The agent will handle the multi-step process of decomposing the question, searching, scraping, and synthesizing the final report automatically.

Code Examples

II-Researcher is designed as a framework. You can integrate its core logic into your own Python applications to build custom research agents.

To run a basic research task programmatically, you can utilize the CLI’s entry point or wrap the agent’s core logic. The following example demonstrates how to trigger a research task via the CLI wrapper:

# Example of running the researcher via CLI in a script
import subprocess

question = "How does the Model Context Protocol (MCP) improve AI agent interoperability?"
subprocess.run(["python", "ii_researcher/cli.py", "--question", question, "--stream"])

For advanced users, the project structure allows you to modify the src/ii_agent/agents/ directory to customize the reasoning loops and the src/ii_agent/tools/ directory to add your own custom search or scraping tools.

Advanced Configuration

II-Researcher provides several flags to customize the output and behavior of the research process through the CLI.

You can control the report style and saving mechanism using these flags:

• --report-type: Set this to basic or advanced to control the depth and structure of the final synthesized report.
• --save-report: When enabled, the agent will automatically save the resulting markdown file to your local disk for later review.

Additionally, you can configure the LLM provider via LiteLLM, allowing you to switch between OpenAI, Anthropic, or local models via OpenRouter, simply by changing the model name in your configuration.

Real-World Use Cases

II-Researcher is most effective when the answer to a question cannot be found in a single source or requires the synthesis of multiple conflicting reports.

• Market Analysis: A product manager can use II-Researcher to compare the feature sets of five different competitors, scraping their pricing pages and documentation to create a comprehensive comparison matrix.
• Academic Literature Review: A researcher can use the tool to find the latest peer-reviewed papers on a specific niche topic, synthesizing the current state of the art and identifying knowledge gaps in the latest publications.
• Technical Due Diligence: A developer can use the agent to research the best library for a specific technical requirement, analyzing documentation, GitHub issues, and Reddit threads to determine the best choice based on real-world usage.
• Content Creation: A technical writer can use the agent to generate a first draft of a deep-dive article, ensuring all claims are grounded in real-time web data and provided with accurate citations.

Contributing to II-Researcher

As an open-source project, II-Researcher welcomes contributions from the community. You can contribute by submitting bug reports, requesting new features, or submitting pull requests to improve the search and scraping tools.

The project follows the standard GitHub flow: fork the repository, create a feature branch, and submit a PR. If you are new to contributing, look for issues labeled as “good first issue” to get started. All contributors are expected to adhere to the the project’s Code of Conduct to maintain a professional and collaborative environment.

Community and Support

II-Researcher is part of the broader Intelligent Internet ecosystem. You can find official support and discussions through the following channels:

• GitHub Discussions: The primary hub for feature requests and bug reports.
• Official Blog: Visit the Intelligent Internet blog for design stories and evaluation details.
• X (Twitter): Follow @ii_posts for updates on the project’s development.
• Discord: Join the Intelligent Internet community Discord for real-time support and support channels.

Conclusion

II-Researcher represents a shift toward transparent, agentic research. By automating the multi-step process of searching, reading, and reflecting, it provides a powerful alternative to proprietary search agents that operate as black boxes. For developers and researchers who require verifiable, grounded data, it is the right choice when the depth of the answer is more important than the speed of a chat response.

While the project is early-stage and requires the user to provide their own API keys, the flexibility and transparency it provides are invaluable for professional-grade research. We recommend starting with the CLI quickstart to experience the reasoning loop in action.

Star the repo, try the quickstart, and join the community to help shape the future of open-source deep research.

Resources

• II-Researcher GitHub Repository
• Official II-Researcher Design Story
• Intelligent Internet X Account