llama.cpp: LLM and VLM inference in C/C++

llama.cpp is a C/C++ implementation of large language model and vision language model inference, built on the ggml tensor library and MIT licensed. It ships a command line interface, an OpenAI-compatible API server, and a web chat UI, installable as an app, via Docker, from pre-built binaries, or from source. Backends cover Apple Silicon with ARM NEON, Accelerate and Metal, x86 with AVX through AVX512 and AMX, NVIDIA CUDA, AMD HIP, Vulkan and SYCL, plus hybrid CPU and GPU inference. The repository showed 124.9k stars and 22.0k forks with 778 open issues and roughly 1.4k open pull requests, and quantization spans 1.5-bit to 8-bit so output quality depends on the level chosen.