Introduction

The efficiency of Large Language Model (LLM) inference on AMD hardware has historically been limited by cross-platform abstractions that fail to exploit specific architectural advantages of RDNA and CDNA cores. While the standard llama.cpp project provides excellent broad support, users of AMD Strix Halo, Ryzen AI Max, and Radeon cards often find themselves hitting performance ceilings with traditional K-quants. ROCmFPX is a specialized, performance-optimized fork of llama.cpp designed specifically for the AMD ROCm ecosystem. By introducing “AMD-first” weight formats and hand-tuned kernels, ROCmFPX delivers up to a 30% increase in prompt processing speeds compared to upstream implementations. This post serves as the primary resource for developers and AI enthusiasts looking to maximize their AMD-based LLM workflows through native ROCmFP quantization.

What Is ROCmFPX?

ROCmFPX is an advanced LLM inference engine that primary functions as a performance-critical fork of llama.cpp for [target user] AMD GPU owners and AI infrastructure developers. Developed by the researcher charlie12345, the project integrates custom floating-point quantization types—specifically ROCmFP3, ROCmFP4, ROCmFP6, and ROCmFP8—directly into the GGUF model format. Unlike standard quantization methods that prioritize compatibility across multiple hardware vendors, ROCmFPX utilizes native accelerated paths for AMD’s gfx1151 (Strix Halo) and RDNA architectures.

The project consists of a core C++ library and a headless OpenAI-compatible server that leverages the ROCm and Vulkan backends. According to the repository’s documentation, these custom quant types (identified by enum IDs 110–115) are real model-weight quants rather than simple K/V-cache compression. This architectural choice ensures that the model preserves high fidelity (perplexity) while significantly reducing the computational overhead during the prefill phase. The project is maintained as an experimental feature family on its main branch, providing early access to next-generation AMD optimizations before they reach upstream stability.

Why ROCmFPX Matters

The gap between theoretical hardware performance and actual LLM throughput is often caused by the “compute-bound” nature of the prompt processing (prefill) phase. Standard GGUF formats like Q6_K use bit-packing strategies that require extra cycles to unpack on AMD hardware. ROCmFPX matters because it replaces these generic formats with native ROCmFP types that the hardware can process more efficiently. In real-world benchmarks on Strix Halo hardware, models quantized with the ROCmFPX format show a measurable 25-30% improvement in prompt tok/s over the stock Q6_K baseline, even when the resulting file size is slightly larger.

Beyond raw speed, ROCmFPX addresses the specific needs of “Agentic” AI. The project includes specialized “AGENT” tiers and support for Multi-Token Prediction (MTP) speculative decoding. For developers building autonomous assistants, this means the difference between an agent that takes 2 seconds to “think” and one that responds almost instantly. By focusing on the nuances of AMD’s unified memory and LPDDR5X bandwidth on systems like Strix Halo, ROCmFPX allows consumer-grade laptops to perform at levels previously reserved for dedicated server-grade clusters. It effectively democratizes high-speed, local LLM serving for the growing ecosystem of AMD users.

Key Features

• Native ROCmFP Quantization: Introduces ROCmFP3 through ROCmFP8 as first-class weight formats (Enum IDs 110-115) specifically tuned for AMD hardware.
• Strix Halo (gfx1151) Optimization: Features hand-written kernels and build scripts targeted at the next-generation Strix Halo and RDNA3 architectures.
• Prefill Speed Acceleration: Delivers up to 30% faster prompt processing (prefill) compared to standard llama.cpp K-quants at equivalent quality levels.
• Multi-Token Prediction (MTP): Supports advanced speculative decoding using MTP heads, which can nearly double generation speeds in real-world serving scenarios.
• Dual-Backend Support: Operates natively via ROCm for Linux/Windows and includes an optimized Vulkan path for hardware where ROCm drivers are not available.
• Headless OpenAI Server: Built-in llama-server implementation that provides a drop-in replacement for OpenAI API endpoints in local agentic scaffolds.
• Advanced Perplexity Maintenance: ROCmFPX quants land within 0.2-0.3% of stock Q6_K perplexity, ensuring no noticeable loss in model intelligence despite the speed gains.
• Experimental AGENT Tiers: Includes specialized model variants and routing logic for tool-use and agentic tasks, optimized for the Hermes-style smoke tests.

How ROCmFPX Compares

In the landscape of local inference engines, ROCmFPX stands out as the only project offering native AMD-optimized floating-point quantization. While standard llama.cpp is the foundation, it must remain generic enough to run on everything from Raspberry Pis to H100s. ROCmFPX sheds this universal baggage in favor of hardware-specific performance. The table below illustrates the throughput gains observed on AMD Strix Halo hardware using the Qwopus 27B model family.

Detailed analysis of these figures reveals a significant nuance: while the ROCmFPX file size is slightly larger (22.39 GiB vs 20.88 GiB for 27B Q6), the compute-bound prefill phase is much faster because the kernels are perfectly aligned with the gfx1151 ISA. The decode phase remains bandwidth-bound, meaning standard generation speeds are comparable to Q6_K at equal sizes unless MTP is engaged. For users on ROCm-supported hardware, this effectively provides a “free” 30% performance boost for prompt processing. However, the primary tradeoff is compatibility; ROCmFPX quants will not load in stock llama.cpp, LM Studio, or Ollama without the specific patches included in this fork.

Getting Started: Installation

Setting up ROCmFPX requires a Linux or Windows environment with appropriate AMD drivers. For the best performance, building from source is mandatory to ensure the kernels are compiled for your specific GPU architecture.

Step 1: Clone the Repository

git clone https://github.com/charlie12345/ROCmFPX.gitncd ROCmFPX

Step 2: Build for Strix Halo / RDNA3

The project provides specialized build scripts for different hardware tiers. For the flagship Strix Halo experience, use the MTP-optimized script.

# Install build dependencies first (cmake, g++, rocm-dev)nJOBS=16 scripts/build-strix-rocmfp4-mtp.sh

Step 3: Post-Build Verification

Verify that the binary has been created in the output folder (typically build-strix-rocmfp4/). You can check the version and available quants by running:

./build-strix-rocmfp4/bin/llama-cli --version

How to Use ROCmFPX

Using ROCmFPX is similar to standard llama.cpp, but with the added requirement of using ROCm-specific flags. The most common use case is running the llama-server to provide an API for external applications.

A typical workflow involves launching the server with the specific GGUF model path. If you are running on Strix Halo hardware, you may need to override the GFX version to ensure the ROCm runtime correctly identifies the accelerator. The server supports multi-GPU offloading via the -ngl flag and should be used with the FlashAttention (-fa on) flag for optimal results. Once the server is running, it exposes a local endpoint that adheres to the OpenAI chat completions schema, allowing you to use it with any compatible agentic framework or UI frontend.

Code Examples

The following example demonstrates how to serve a flagship 27B model using the ROCmFPX engine with MTP speculative decoding enabled. This command is optimized for the Strix Halo architecture.

# Serve the Qwen 27B flagship model with MTP and Vision supportnHSA_OVERRIDE_GFX_VERSION=11.5.1./build-strix-rocmfp4/bin/llama-server n -m Qwen3.6-27B-STRIX-embF16-headQ6-Q6_0_ROCMFPX.gguf n -dev ROCm0 -ngl 999 -fa on -c 32768 n --spec-type draft-mtp --spec-draft-ngl all --spec-draft-n-max 2 n --jinja --mmproj mmproj/

For users who prefer a simpler CLI-based interaction for benchmarks, the following command shows how to run a prompt through the local inference engine:

./build-strix-rocmfp4/bin/llama-cli n -m your-model-ROCMFPX.gguf n -p "Explain the advantages of AMD ROCm quantization." n -n 128 -ngl 999 -fa on

Advanced Configuration

For power users, ROCmFPX offers deep configuration via environment variables and specific CLI arguments. The HSA_OVERRIDE_GFX_VERSION=11.5.1 variable is crucial for Strix Halo users to bypass driver-level version mismatches. Additionally, you can tune the --spec-draft-n-max parameter to control the number of speculative tokens; while 2 is standard for MTP, some tasks benefit from higher values at the cost of slight compute overhead. For those running on Windows, the project supports a native build via PowerShell scripts located in the scripts/ directory, though the Linux ROCm path remains the recommended route for maximum performance. If you are experiencing slower than expected prefill on MoE (Mixture of Experts) models, ensure that your kernels are properly compiled for the specific expert tensor types used in that architecture.

Real-World Use Cases

• Strix Halo Laptops: Transform high-end Ryzen AI Max laptops into mobile AI workstations capable of serving 27B+ models at interactive speeds without an external GPU.
• Local Agentic Scaffolds: Power complex multi-turn agents that require fast prefill to process large codebase contexts or long conversation histories.
• Private Knowledge Bases: Host secure, local LLMs on Radeon-equipped workstations to summarize sensitive documents using the native ROCmFP8 formats.
• Scientific Research: Utilize the 0.2% perplexity precision of ROCmFPX for academic research where model accuracy is as important as inference speed.
• Vulkan-Only Environments: Deploy high-performance GGUF inference on systems where proprietary ROCm drivers cannot be installed, utilizing the project’s optimized Vulkan path.

Contributing to ROCmFPX

The ROCmFPX project is actively seeking contributors to help expand its coverage of AMD architectures and optimize MoE expert kernels. According to the AGENTS.md and CONTRIBUTING.md guidelines, the maintainers specifically value PRs that include benchmark data comparing the fork against upstream llama.cpp. If you discover a bug in the custom quant loaders or wish to add support for a new RDNA generation, you can open an issue on the GitHub repository. Contributors are reminded to follow the project’s “handoff” protocol, ensuring that all changes are appropriately scoped and follow the existing coding conventions. For major architectural changes, it is recommended to discuss the approach in the Issues section before submitting a pull request to ensure alignment with the project’s performance-first philosophy.

Community and Support

Official support for ROCmFPX is managed primarily through the GitHub repository’s Issues and Discussions tabs. Users can also find active technical deep-dives and benchmark reports on the project’s Reddit community and the Hugging Face model cards maintained by @philtheriver. For direct technical documentation, the docs/ folder in the repo contains a wealth of information, including ROCmFPX-HANDOFF.md for reviewers and BUILD-AMD-ARCHITECTURES.md for specific RDNA/Strix setup details. Following the developer charlie12345 on GitHub is the best way to stay informed about new experimental branches and performance updates for next-gen AMD hardware.

Conclusion

ROCmFPX represents a vital shift in the LLM ecosystem toward hardware-aware optimization. By providing native AMD floating-point quantization and hand-tuned kernels for Strix Halo and Radeon architectures, it eliminates the performance tax of generic inference engines. Whether you are an enthusiast running local agents on a Ryzen AI laptop or a developer building enterprise AI on Radeon clusters, ROCmFPX offers the speed and precision required for professional-grade performance. While the project remains experimental, its 30% prefill speedup and native MTP support make it a compelling choice for anyone committed to the AMD AI stack.

We recommend starting with the 27B flagship models on Hugging Face to see the speed difference for yourself. If you are building on AMD, ROCmFPX is not just an alternative—it is the definitive performance standard. Star the repository, benchmark your hardware, and join the community of developers who are making fast, local AI a reality on AMD hardware. The future of inference is native, and ROCmFPX is leading the way for AMD users.

Resources

• ROCmFPX GitHub Repository
• Optimized ROCmFPX GGUF Models on Hugging Face
• Official AMD ROCm Documentation
• Upstream llama.cpp Project