AI at Home Part 2: Multi-GPU Drifting

A hands-on technical writeup on running LLM inference across multiple AMD Radeon Pro V620 GPUs in a home server. The author found that layer parallelism hurt performance, dropping from 19-20 tokens per second on a single card to 12-13 across four cards, while speculative decoding roughly doubled speeds to about 40 tokens per second on a single card. PCIe 3.0 with 8 lanes created bottlenecks for inter-GPU communication, with peer-to-peer transfers and BIOS tuning improving tensor parallelism across two cards. Models tested include Gemma4-31B and Deepseek V4 Flash.