Multiport Support for Vortex OpenGPU Memory Hierarchy

Abstract: Modern day applications have grown in size and require more computational power. The rise of machine learning and AI increased the need for parallel computation, which has increased the need for GPGPUs. With the increasing demand for computational power, GPGPUs' SIMT architecture has solved this with an increase in the number of threads and the number of cores in a GPU, increasing the throughput of these processors to match the demand of the applications. However, this created a larger demand for the memory, making the memory bandwidth a bottleneck. The introduction of High-Bandwidth Memory (HBM) with its increased number of memory ports offers a potential solution for the GPU to exploit its memory parallelism to increase the memory bandwidth. However, effectively leveraging HBM's memory parallelism to maximize bandwidth presents a unique and complex challenge for GPU architectures on how to distribute those ports among the streaming multiprocessors in the GPGPU. In this work, we extend the Vortex OpenGPU microarchitecture to incorporate a multiport memory hierarchy, spanning from the L1 cache to the last-level cache (LLC). In addition, we propose various arbitration strategies to optimize memory transfers across the cache hierarchy. The results have shown that an increase in memory ports increases IPC, achieving an average speedup of 2.34x with 8 memory ports in the tested configuration while showing relatively small area overhead.