Using neuromorphic sparsity for perception and AI inference

Activity-driven computation is key to brain power efficiency. The sparse, rapid output from neuromorphic event cameras enables faster vision systems that consume less power and operate effectively under challenging lighting conditions. I will demonstrate an event camera, then discuss how its sparse output inspired several generations of neural accelerators (developed within the Samsung Neuromorphic Processor global research project). These accelerators exploit various forms of dynamic sparsity to operate faster and more efficiently than conventional approaches, while retaining the compact area and high throughput of traditional neural accelerators. Spiking neural networks (SNNs) are popular in the neuromorphic community, but they are fundamentally incompatible with the requirement for abundant, fast, and cost-effective memory for states and weights. The convolutional and recurrent deep neural network (DNN) hardware accelerators I will present exploit spatial and temporal sparsity, similar to SNNs. However, they achieve state-of-the-art throughput, power efficiency, area efficiency, and low latency while utilizing DRAM for the large weight and state memory required by powerful DNNs. I will summarize how some of these concepts appear in the latest mass production Samsung application processor neural processing unit.