DNN-SAM

Split-and-merge DNN execution for timely and accurate object detection.

Split-and-merge DNN execution for real-time object detection

Multi-camera real-time object-detection systems must balance accuracy and timeliness across image regions with different criticality. DNN-SAM dynamically splits an inference task into independently scheduled subtasks:

  • A mandatory subtask processes a cropped, safety-critical region.
  • An optional subtask processes a down-scaled version of the complete image.
  • A merge stage combines their detections into the final result.
DNN-SAM pipeline splitting an image into mandatory and optional networks, scheduling their subtasks on GPUs, and merging their detection results
DNN-SAM system overview: split, independently schedule, and merge mandatory and optional DNN subtasks.

The scheduler prioritizes subtasks according to criticality and adapts input scale to meet timing constraints. In evaluation, DNN-SAM:

  • Improved detection accuracy in safety-critical regions by 2.0-3.7x.
  • Reduced average inference latency by 4.8-9.7x.
  • Met all evaluated timing constraints.
Emergency-braking case-study comparison showing DNN-SAM stopping within the safety distance while the baseline exceeds it
Emergency-braking case study comparing the baseline with DNN-SAM.

DNN-SAM was published at IEEE RTAS 2022.

Demo video

DNN-SAM case study using a 1/10-scale autonomous car.

References

2022

  1. RTAS
    DNN-SAM: Split-and-Merge DNN Execution for Real-Time Object Detection
    Woosung Kang, Siwoo Chung, Jeremy Yuhyun Kim, Youngmoon Lee, Kilho Lee, Jinkyu Lee, Kang G. Shin, and Hoon Sung Chwa
    In 2022 IEEE 28th Real-Time and Embedded Technology and Applications Symposium (RTAS), 2022
    45 Google Scholar citations