All Meta AI items under Companies on AIFIRST News, newest first. 10 items.
GLM-4.5-Air, a 106B mixture-of-experts model with only 12B active parameters, now supports Multi-Token Prediction in llama.cpp for a meaningful inference speed boost. The update makes it more practical for users who want a large MoE without requiring multiple high-end GPUs. MTP support is enabled via a configuration flag in the updated build.
The ConvRot quantization method, which outperforms standard UDQ8_K_XL in accuracy benchmarks, has been integrated into the llama-cpp-turboquant fork. ConvRot applies a learned rotation to model weights before quantization, reducing the accuracy loss from low-bit formats. The fork is available on GitHub for users who want better quantization quality beyond standard GGUF methods.
A developer ran a 3-day benchmark of Inco AI's DFlash 2 speculative decoding PR in llama.cpp against plain decoding, MTP, n-gram lookup drafters, and the previous DFlash 1 on Qwen 3.8 27B. DFlash 2 introduces a trained drafter specifically for Qwen 3.8 27B and shows meaningful throughput improvements over all comparison methods.
A developer released a llama.cpp fork with HIP optimizations targeting AMD GFX906-class GPUs including the Mi50, Mi60, Radeon VII, and GCN HIP cards. These cards have minimal support in the mainline llama.cpp codebase, leaving many AMD users without an optimized inference path.
nRouter SDK puts a single API key and one interface in front of six cloud model providers: Alibaba US, OpenAI, AWS Bedrock, Azure Foundry, Google Vertex AI, and Anthropic. It speaks both the OpenAI and Anthropic wire formats natively and covers chat completions, embeddings, images, text to speech, speech to text, and video. Branded packages ship for Python, JavaScript, Java, Rust, and R, with OpenAI-compatible paths for Go, Ruby, PHP, and C#. Framework integrations include LangChain, LlamaIndex, Vercel AI SDK, CrewAI, and AutoGen.
llama.cpp is a C/C++ implementation of large language model and vision language model inference, built on the ggml tensor library and MIT licensed. It ships a command line interface, an OpenAI-compatible API server, and a web chat UI, installable as an app, via Docker, from pre-built binaries, or from source. Backends cover Apple Silicon with ARM NEON, Accelerate and Metal, x86 with AVX through AVX512 and AMX, NVIDIA CUDA, AMD HIP, Vulkan and SYCL, plus hybrid CPU and GPU inference. The repository showed 124.9k stars and 22.0k forks with 778 open issues and roughly 1.4k open pull requests, and quantization spans 1.5-bit to 8-bit so output quality depends on the level chosen.
Vomit is a small open-source tool that intercepts Claude agent mode's intermediate reasoning output and pipes it through a local LLM to produce more readable English summaries. It runs fully locally with no telemetry, supports Llama.app and Ollama backends, and can be used either as an integrated hook replacing Claude's displayed output or as a standalone session monitor via the 'vomit tail' command. The tool targets the verbosity of Claude's token output during agentic tasks.
Shoehorn is a locally run, browser-based tool that quantizes language models with hardware-aware mixed precision, assigning a precision level per tensor rather than applying a preset and targeting about 99.99% of available memory. It scans popular Hugging Face models to work out what fits a given machine and writes standard GGUF v3 files for llama.cpp. It supports macOS on Apple Silicon, Linux on x86-64, and Windows on x86-64. The quantizer core is written from scratch in Rust.
LLaMA-Factory provides a unified web UI + CLI for fine-tuning LLMs and VLMs without writing custom training code. Supports LLaMA, Mistral, Qwen, DeepSeek, Gemma, Phi, Yi, and 90+ others. Positioned as a no-code altern...
LlamaIndex's Jerry Liu is open sourcing ParseBench - a benchmark specifically measuring document parsing quality for AI agent workflows. The argument: current OCR/parsing benchmarks don't reflect real agentic use case...
AIFIRST 
