Filtered by: GPU × Clear all

ELiTeFormer: An Efficient Transformer for FPGAs

Victor Agostinelli, Nicolas Bohm Agostini, Antonino Tumeo 2026-07-07
Attention Language Model LLM Quantization Transformer Fpga GPU ×

ELiTeFormer addresses the deployment challenge of Transformer blocks in LLMs by co-designing hybrid linear attention with ultra-low-precision ternary projections specifically for FPGAs. The method introduces a novel processing element micro-architecture that eliminates all multiplications in ternary linear projections using bitmasking, avoiding DSP blocks entirely. Experimental results show 10x model weight compression and 12.8x KV cache compression versus LLaMA 3, with block-level simulations achieving 9.6x speedup for FFN and 4.4x for attention, and end-to-end deployment delivering up to 3.9x lower latency and 3.2x better energy efficiency than an NVIDIA A100 GPU. This matters as it demonstrates the first FPGA realization combining linear attention with ternary quantization, proving algorithm-architecture co-design viability for next-generation LLM acceleration.

PDF

Direct Model State Migration for Elastic Training of Large Language Models

Weijian Liu, Mingzhen Li, Rui Kang, Chen Sun 2026-07-07
Language Model LLM Training GPU ×

The problem is that elastic training of large language models (LLMs) requires state migration across device sets when hybrid-parallel configurations change, but existing checkpoint-based solutions force all GPUs to stall and incur prohibitive latency from data movement across memory hierarchies. ETC proposes a checkpoint-free state migration framework that exploits state locality to minimize inter-GPU data movement, replacing storage persistence with direct peer-to-peer communication and eliminating node fragmentation through communication coalescing. Integrated with Megatron-LM, ETC reduces migration overhead by 2.33× to 6.37× compared to checkpoint-based solutions across diverse parallel configurations. This matters because ETC unlocks practical elastic training in production environments by enabling efficient migration, allowing LLM training to adapt to dynamic resources in shared clusters.

PDF

GORIO: GPU-Centered Remote I/O for Graph ANNS over NVMe-oF

Gen Zhang, Wenhao Gu 2026-07-07
GPU × Search Data Design Cache Scheduling Workload

GORIO addresses the problem that graph-based approximate nearest neighbor search (ANNS) vector indexes often exceed GPU memory, and existing CPU-centered remote I/O over NVMe-oF is poorly matched to GPU graph traversal. The method keeps all query evolution, page-miss generation, and resume decisions on the GPU, using the CPU only as an NVMe-oF transport proxy, with a two-layer design for GPU-direct remote I/O and ANNS-specific scheduling. On a SIFT1M DiskANN-style workload over RDMA NVMe-oF, GORIO achieves 1.31× speedup over the state-of-the-art remote-I/O reference and 3.73× over the direct remote page-cache path. This matters because it provides a concrete GPU-centered remote I/O substrate that significantly accelerates graph ANNS for vector databases and retrieval-augmented generation services.

PDF