Filtered by: Inference × Clear all

Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing

Sadegh Majidi, Niloofar Mireshghallah, Kazem Taram 2026-07-24
Attention Inference × Language Model Transformer Hardware Architecture

LeakyLMs introduces a set of attacks that leak proprietary model architecture and deployment information from production language models using only per-token generation timing. The method builds a detailed timing model of token generation on NVIDIA GPUs and performs a search over the architecture space to infer properties like transformer layers, hidden dimension size, and attention heads. Experimental evidence shows that for Llama models, the near-correct architectural configuration appears in the top-10 guesses over 90% of the time, and the attack successfully detects speculative decoding in Google Gemini Flash 2.5 with a draft context window of approximately 128K tokens. This matters because it demonstrates that sensitive model and deployment details can be stolen remotely via timing side channels, posing a significant security risk to proprietary language model services.

PDF

A Flexible Sparsity-Aware FPGA Accelerator with Column-Wise Compression for Efficient CNN Inference

Amirhossein Zarei, Shervin Vakili 2026-07-24
Microarchitecture Simulation Sparsity Accelerator Inference ×

Problem: Efficient CNN acceleration on resource-constrained FPGAs is challenged by the irregularity of sparsity patterns and associated hardware overhead. Method: SparHiXcel-v2 introduces a column-wise kernel compression technique within a scalable 2D MAC array and a hardware-algorithm co-design framework with ordering optimization and multi-phase structured pruning. Finding: On a cost-effective AMD Kintex UltraScale+ FPGA, SparHiXcel-v2 achieves over 2.5 TOPS and 210 GOP/s/W for VGG16 and over 1.1 TOPS and 72 GOP/s/W for ResNet18 with modest accuracy degradation. Why it matters: This work demonstrates a practical balance between sparsity flexibility and hardware efficiency, enabling high-throughput, energy-efficient CNN inference on resource-constrained platforms.

PDF