Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization

The problem is that LLM serving systems remain memory-intensive and costly due to the KV cache storing tensors during autoregressive decoding. The method organizes existing system-aware KV cache optimization efforts into three dimensions: execution and scheduling (temporal), placement and migration (spatial), and representation and retention (structural). The abstract does not disclose experimental results. This matters because it systematizes a rapidly evolving area, providing a foundation for understanding and innovating KV cache designs in modern LLM serving infrastructure.