ZeroSwap

SSD-based GPU memory extension for real-time multi-DNN inference.

Minimizing swap overhead for real-time multi-DNN inference

Real-time multi-DNN systems on embedded platforms are often constrained by limited GPU memory. Extending memory through SSD swapping can address capacity limits, but PCIe transfers and runtime memory management introduce substantial overhead.

ZeroSwap minimizes that overhead through three mechanisms:

  • Semantic-aware selective swapping moves only inference-critical data, reducing swap volume.
  • Shared pinned allocation removes runtime allocation overhead through physical-memory sharing.
  • Segment-level overlapping hides PCIe transfer latency by overlapping swap operations with computation.
ZeroSwap architecture connecting DNN applications, shared pinned allocation, semantic-aware selective swapping, an overlap-aware scheduler, system RAM, and SSD storage
ZeroSwap design overview: shared pinned allocation, semantic-aware selective swapping, and overlap-aware scheduling.

Implemented on a state-of-the-art machine-learning framework, ZeroSwap improved DNN task-set schedulability by up to 101.7% and achieved up to 3.2x faster response time compared with existing approaches.

Runtime latency breakdown comparing RT-Swap and ZeroSwap, showing shorter execution and swap latency with ZeroSwap
Runtime latency breakdown comparing ZeroSwap with RT-Swap.

ZeroSwap was published at IEEE RTAS 2026 and received the Best Paper Award.

Demo video

ZeroSwap case study comparing multi-camera object-detection performance with RT-Swap.

References

2026

  1. RTAS
    ZeroSwap: Minimizing Swap Overhead for Real-Time Multi-DNN Inference via SSD-based GPU Memory Extension
    Woosung Kang, Filippo Muzzini, Gianluca Brilli, Jong-Chan Kim, Jinkyu Lee, and Hoon Sung Chwa
    In 2026 IEEE 32nd Real-Time and Embedded Technology and Applications Symposium (RTAS). 🏆 Best Paper Award , 2026
    0 Google Scholar citations