ExLlama: High-Performance Llama Inference for NVIDIA GPUs

Introduction ExLlama is a cutting-edge implementation of the Llama model designed for high performance and efficiency on modern GPUs. Utilizing 4-bit GPTQ weights, ExLlama aims to provide developers with a robust tool for deploying Llama models with minimal memory...