Tempus Core: Area-Power Efficient Temporal-Unary Convolution Core for Low-Precision Edge DLAs

Abstract: The increasing complexity of deep neural networks (DNNs) poses significant challenges for edge inference deployment due to resource and power constraints of edge devices. Recent works on unary-based matrix multiplication hardware aim to leverage data sparsity and low-precision values to enhance hardware efficiency. However, the adoption and integration of such unary hardware into commercial deep learning accelerators (DLA) remain limited due to processing element (PE) array dataflow differences. This work presents Tempus Core, a convolution core with highly scalable unary-based PE array comprising of tub (temporal-unary-binary) multipliers that seamlessly integrates with the NVDLA (NVIDIA's open-source DLA for accelerating CNNs) while maintaining dataflow compliance and boosting hardware efficiency. Analysis across various datapath granularities shows that for INT8 precision in 45nm CMOS, Tempus Core's PE cell unit (PCU) yields 59.3% and 15.3% reductions in area and power consumption, respectively, over NVDLA's CMAC unit. Considering a 16x16 PE array in Tempus Core, area and power improves by 75% and 62%, respectively, while delivering 5x and 4x iso-area throughput improvements for INT8 and INT4 precisions. Post-place and route analysis of Tempus Core's PCU shows that the 16x4 PE array for INT4 precision in 45nm CMOS requires only 0.017 mm^2 die area and consumes only 6.2mW of total power. We demonstrate that area-power efficient unary-based hardware can be seamlessly integrated into conventional DLAs, paving the path for efficient unary hardware for edge AI inference.

• The paper presents Tempus Core, which leverages a temporal-unary-binary convolution design to cut area by 75% and power by 62% over traditional architectures.
• It achieves a 5x iso‐area throughput improvement for INT8 and 4x for INT4 precision, demonstrating significant efficiency gains in low‐precision edge DLAs.
• The design integrates with standard dataflows, ensuring seamless compatibility with DLAs like NVIDIA's NVDLA for enhanced edge AI applications.

A Professional Analysis of "Tempus Core: Area-Power Efficient Temporal-Unary Convolution Core for Low-Precision Edge DLAs"

The paper "Tempus Core: Area-Power Efficient Temporal-Unary Convolution Core for Low-Precision Edge DLAs," authored by researchers from Carnegie Mellon University, addresses the growing demands placed on edge devices by deep neural networks (DNNs). The main challenge is the resource and power constraints limiting the adoption of DNNs for edge inference. This research proposes Tempus Core, a temporal-unary-binary based convolution core designed to improve hardware efficiency while maintaining compatibility with existing architectures, specifically the NVIDIA Deep Learning Accelerator (NVDLA).

Overview of the Tempus Core Design

Tempus Core introduces a novel ripple effect in efficiency through its temporal-unary-binary paradigm. By utilizing tub multipliers within a processing element (PE) array, Tempus Core attempts to bridge the efficiency gap observed in traditional binary-based designs. This paper emphasizes Tempus Core's compatibility with standard dataflow methodologies, demonstrating its potential for seamless integration with widely utilized DLAs like NVDLA. The integration allows for considerable reductions in area and power consumption compared to the conventional CMAC unit of NVDLA, quantified at 75% in area and 62% in power for INT8 precision using a $16 \times 16$ PE array configuration.

Numerical and Architectural Insights

Analyzing across various granularities, the paper presents strong quantitative results indicating the potential of unary-based architectures. For instance, the design achieves 5x iso-area throughput improvement for INT8 and 4x for INT4 precision. These results underline a distinct advantage of Tempus Core in terms of both scalability and efficiency. The key architectural innovation lies in the unique implementation of the tub multipliers and a multi-cycle operation strategy, rather than the typical single-cycle approach.

Practical and Theoretical Implications

Theoretically, Tempus Core sets a precedent in designing DLAs that could extend beyond traditional binary constraints, paving the way for more energy-efficient AI computations at the edge. The practical implications are immense, particularly in scenarios constrained by thermal or power budgets, such as portable and embedded AI. The ability to integrate such efficient computations into existing DLAs while maintaining system coherence presents significant industrial value, enhancing overall system throughput without requiring architectural overhauls.

Speculations on Future Developments

Considering the rapid evolution of quantization techniques and the demand for low-power computations, future development could extend Tempus Core's methodologies to ultra-low precision quantized models, such as those employed in LLM deployments. Further, exploring custom dataflows optimized for latency reductions could translate these theoretical gains into attainable real-world benefits.

Overall, the research exemplifies a meticulous combination of architecture redesign and quantization exploitation, offering a compelling solution that enhances the viability of edge AI implementations. The scope for continuous improvement and adaptation to upcoming AI workloads positions Tempus Core as a robust contender in the ongoing development of efficient edge DLAs.