Filtered by: Hardware × Inference × Clear all

CODA: Algorithm-Hardware Co-design for Edge Video Diffusion via NMP-Enabled Compute-Cache Operator Disaggregation

Yuanpeng Zhang, YuXuan Wu, Yitong Xiao, Chenhao Xue 2026-07-18
Inference × Transformer Hardware × Design Cache

The problem is that deploying Video Diffusion Models on edge devices is too slow for practical local inference due to iterative Transformer-based denoising. The method, CODA, uses algorithm-hardware co-design to disaggregate compute and cache operators via a DIMM-side near-memory engine, reorganizing cache activity into coalesced segments and overlapping compute with cache execution. Experimental evidence shows CODA achieves up to 1.80x end-to-end speedup and 1.74x higher energy efficiency while preserving generation quality. This matters because it enables practical, privacy-preserving video generation on memory-constrained edge GPUs by overcoming the communication and serialization bottlenecks of Cross-Timestep Caching.

PDF

Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent

Lingyun Yang, Yuxiao Wang, Shenghao Liang, Linfeng Yang 2026-07-18
Inference × LLM GPU Hardware × Agent

Atrex-Bench addresses the problem that existing GPU kernel benchmarks use synthetic or curated workloads, not production traces. The method samples 30 operators and 440 shapes from full-cluster production inference traces, weighting each by observed GPU time and card-hours. Experimental evidence shows the best vanilla coding agent reaches only ~10% of the hardware roofline on production operators, with much of the pass rate from PyTorch fallbacks. The co-released Atrex-Kernel-Agent (AKA) converts zero-FlyDSL fallbacks into real kernels matching hand-tuned baselines, demonstrating that profile-driven optimization is critical for production readiness.

PDF