Fine-grained Computation-Communication Overlap via Tile-level Signaling and Scheduling for Mixture-of-Experts

Minyu Cui, Anna Wingkvist, Morgan Ericsson 2026-07-24
Language Model GPU Communication Scheduling

The problem is that conventional Mixture-of-Experts (MoE) implementations launch the return all-to-all communication after expert compute completes, exposing communication latency on the critical path and reducing GPU utilization. The method introduces a fine-grained producer-consumer co-design with tile-level signaling and scheduling, using a persistent per-rank computation kernel and a persistent communication kernel on a small dedicated SM partition to overlap expert compute with the second all-to-all. On a 4-A100 GPU platform, evaluated on three MoE models against four state-of-the-art MoE systems, the approach achieves up to 2.64x end-to-end speedup and 2.74x MoE-layer speedup. This matters because it provides a practical, non-intrusive way to improve distributed MoE execution efficiency on multi-GPU systems, enabling better scaling of large language models.

PDF

DGNA: Dissecting GPU NUMA Architecture through Microbenchmarking and Data Analysis

Changxi Liu, Yun Chen, Trevor E. Carlson 2026-07-24
Cache Coherence Memory Hierarchy Numa Microarchitecture Simulation HPC

The problem is that modern GPU memory architectures, particularly NUMA mechanisms within L2 and DRAM, remain black-boxes, hindering optimization and simulation. DGNA introduces a methodology using microbenchmarking and Gaussian mixture models to measure L2 and DRAM latency without relying on intrinsic instructions. Applied to NVIDIA A100 and H100 GPUs, it reveals NUMA node architecture, SM-NUMA relationships, and coherence-aware allocation strategies. This matters because it is the first work to detail GPU memory subsystem NUMA, enabling better application tuning and architectural design.

PDF

Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation

Quazi Ishtiaque Mahmud, Nesreen K. Ahmed, Ali Jannesari 2026-07-24
CUDA LLM PyTorch Agent

CudaPerf addresses the problem that existing RLVR methods for CUDA kernel generation overlook structural code properties like memory coalescing and occupancy, relying only on outcome-based signals. The method proposes a two-stage reflective RL framework with offline pairwise ranking and online RL training that jointly optimizes correctness, performance, and structural efficiency via a unified reward. Experimental evidence shows CudaPerf achieves up to 5X and 3.32X speedup improvements and 17% and 7% correctness gains over Qwen-3-32B and CUDA Agent on C-to-CUDA and PyTorch-to-CUDA benchmarks. This matters because it demonstrates that incorporating structural code-aware rewards significantly enhances LLM-driven optimized code generation for high-performance computing.

PDF

Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing

Sadegh Majidi, Niloofar Mireshghallah, Kazem Taram 2026-07-24
Attention Inference Language Model Transformer Hardware Architecture

LeakyLMs introduces a set of attacks that leak proprietary model architecture and deployment information from production language models using only per-token generation timing. The method builds a detailed timing model of token generation on NVIDIA GPUs and performs a search over the architecture space to infer properties like transformer layers, hidden dimension size, and attention heads. Experimental evidence shows that for Llama models, the near-correct architectural configuration appears in the top-10 guesses over 90% of the time, and the attack successfully detects speculative decoding in Google Gemini Flash 2.5 with a draft context window of approximately 128K tokens. This matters because it demonstrates that sensitive model and deployment details can be stolen remotely via timing side channels, posing a significant security risk to proprietary language model services.

PDF

A Flexible Sparsity-Aware FPGA Accelerator with Column-Wise Compression for Efficient CNN Inference

Amirhossein Zarei, Shervin Vakili 2026-07-24
Microarchitecture Simulation Sparsity Accelerator Inference

Problem: Efficient CNN acceleration on resource-constrained FPGAs is challenged by the irregularity of sparsity patterns and associated hardware overhead. Method: SparHiXcel-v2 introduces a column-wise kernel compression technique within a scalable 2D MAC array and a hardware-algorithm co-design framework with ordering optimization and multi-phase structured pruning. Finding: On a cost-effective AMD Kintex UltraScale+ FPGA, SparHiXcel-v2 achieves over 2.5 TOPS and 210 GOP/s/W for VGG16 and over 1.1 TOPS and 72 GOP/s/W for ResNet18 with modest accuracy degradation. Why it matters: This work demonstrates a practical balance between sparsity flexibility and hardware efficiency, enabling high-throughput, energy-efficient CNN inference on resource-constrained platforms.

PDF

A Reinforcement-Learning-Augmented Liquid-Fueled Reactor Network Model for Predicting Lean Blowout in Gas Turbine Combustors

Philip John, Eloghosa Ikponmwoba, Pinaki Pal, Opeoluwa Owoyele 2026-07-24
Design Space Exploration Agentic Architecture Network

The problem is that existing methods for determining reactor cluster boundaries in lean blowout (LBO) prediction rely on manual heuristics or distance-based metrics. The method introduces a reinforcement learning (RL) framework with a multi-stage clustering–classification strategy, using an actor-critic agent to merge micro-clusters into optimal reactor zones. Experimental evidence using a Jet-A mechanism shows the RL framework improves predictive fidelity over k-means and captures correct LBO trends with substantial speedups. This matters because the RL-driven approach offers a computationally efficient reduced-order model for rapid design-space exploration in gas turbine combustors.

PDF

DMG: A Scalable and Efficient Memory-Disaggregated Graph Processing System

Yizou Chen, Tsun-Yu Yang, Zhisheng Hu, Baotong Lu 2026-07-24
Data Architecture Cache Workload Runtime Scheduling

Traditional graph processing systems on monolithic servers suffer from resource under-utilization, and existing disaggregated memory (DM) systems are impractical due to poor scalability and excessive compute-side cache demands. DMG introduces a DM-friendly graph store, an adaptive update coordinator, and a two-stage workload manager to address these issues. Experiments show DMG achieves up to 4.9X better performance and reduces cache demands by up to 18.9X compared to state-of-the-art DM-based systems. This matters because DMG is the first practical, scalable, and cache-efficient graph processing system for DM architectures, enabling efficient handling of growing graph sizes in data centers.

PDF