Exploiting Mixed Precision in Numerical Linear Algebra

October 29, 2021, zoom

Erin Carson

Abstract

Support for floating point arithmetic in multiple precisions is becoming increasingly common in emerging architectures. Mixed precision capabilities are already included in many machines on the TOP500 list and are expected to be a crucial hardware feature in coming exascale machines. From a computational scientists perspective, our goal is to determine how and where we can exploit mixed precision computation in our codes. This requires both an understanding of performance characteristics as well as an understanding of the numerical behavior of algorithms in finite precision arithmetic. In this talk, we discuss recent and ongoing efforts in this area. In particular, we present and analyze a general algorithm for solving nxn nonsingular linear systems Ax = b based on iterative refinement in three precisions. From this, we develop GMRES-IR, a three-precision GMRES-based iterative refinement scheme that works for even ill-conditioned systems. We discuss performance results on modern GPU architectures and present the HPL-AI benchmark, based on our mixed precision iterative refinement algorithms. The world's top supercomputers already exceed exaflop performance on HPL-AI, achieving over 4x higher performance than on the standard HPL benchmark.

Speaker's Bio

Erin Carson is a research scientist at Charles University. She received a Ph.D. in Computer Science in 2015 from the
University of California-Berkeley. Her research focuses on numerical linear algebra, parallel algorithms, and high performance computing.