Filtered by: Design × Inference × Clear all

CODA: Algorithm-Hardware Co-design for Edge Video Diffusion via NMP-Enabled Compute-Cache Operator Disaggregation

Yuanpeng Zhang, YuXuan Wu, Yitong Xiao, Chenhao Xue 2026-07-18
Inference × Transformer Hardware Design × Cache

The problem is that deploying Video Diffusion Models on edge devices is too slow for practical local inference due to iterative Transformer-based denoising. The method, CODA, uses algorithm-hardware co-design to disaggregate compute and cache operators via a DIMM-side near-memory engine, reorganizing cache activity into coalesced segments and overlapping compute with cache execution. Experimental evidence shows CODA achieves up to 1.80x end-to-end speedup and 1.74x higher energy efficiency while preserving generation quality. This matters because it enables practical, privacy-preserving video generation on memory-constrained edge GPUs by overcoming the communication and serialization bottlenecks of Cross-Timestep Caching.

PDF

NIFA: Nonlinear IMC enhanced FPGA for efficient ML inference

Jiajun Hu, Ruthwik Reddy Sunketa, Lei Zhao, Archit Gajjar 2026-07-18
Design × Space Exploration Agentic Architecture Inference ×

The problem is that conventional ReRAM-based analog in-memory computing (IMC) blocks in FPGAs only support static-weight vector-matrix multiplication, limiting efficiency gains for Transformer models that require frequent nonlinear and dynamic matrix-matrix multiplication (DIMM) operations, while the ADCs within each IMC block consume over 70% of its area and power. The method proposes a novel FPGA architecture integrating an ADC-free IMC block that replaces ADCs with analog content-addressable memories (ACAMs) to natively perform nonlinear operations, along with FPGA-aware design-space exploration and efficient mapping to enable DIMM operations for attention computation. Experimental evidence shows that on CNN and Transformer-based benchmarks, the architecture achieves up to 40x and 1.9x higher energy efficiency and 4.1x and 2.5x higher area efficiency, respectively. This matters because it significantly improves FPGA deep learning inference efficiency, particularly for Transformer-based workloads across long input sequences, advancing domain-specialized FPGA design.

PDF