borG: Cross-accelerator coding for fast batched kernel generation¶
Daniel Thuerck (NEC Laboratories Europe)
Abstract¶
A large portion of the recent performance increase in the High Performance Computing and Machine Learning domains is fueled by accelerator cards. As popular cloud compute platforms show, compute platforms are becoming more and more heterogeneous, raising once again the question of simplicity in software development. As an abstraction, many popular ML frameworks support accelerators by organizing computations as a computational graph over a set of highly optimized, batched general-purpose kernels. While this approach simplifies the kernels' implementation for each individual accelerator, the increasing heterogeneity among accelerator architectures for HPC complicates the creation of portable and extensible libraries of such kernels. Therefore, using a generalization of the CUDA community's warp-register cache programming idiom, we propose a new programming idiom (CoRe) and a virtual architecture model (PIRCH), abstracting over SIMD and SIMT paradigms. We define and automate the mapping process from a single source to PIRCH's intermediate representation and develop backends that issue code for three different architectures: Intel AVX512, NVIDIA GPUs, and NEC SX-Aurora. Code generated by our source-to-source compiler for batched kernels, borG, competes favorably with vendor-tuned libraries and is up to 2x faster than hand-tuned kernels across architectures.