SiFAR: Synchronization-Free All-Reduce for Low-Latency LLM Inference

The problem is that LLM token-generation latency, critical for reasoning models and agentic systems, is bottlenecked by All-Reduce overhead in Tensor Parallelism, which grows with GPU count. The method, Synchronization-Free All-Reduce (SiFAR), eliminates synchronization barriers through dual buffering, redundant pull using in-switch reduction, and speculative reduction. Experimental evidence shows SiFAR reduces All-Reduce latency by up to 52% and improves end-to-end throughput by 18.6% for Llama-3.1-8B and 13.1% for Qwen3.5-397B-17B at TP=8 on H200 GPUs. This matters because it enables low-latency LLM inference at scale without the overhead of traditional All-Reduce synchronization.