One-Minute Derivation of The Conjugate Gradient Algorithm

Abstract

One of the great triumphs in the history of numerical methods was the discovery of the Conjugate Gradient (CG) algorithm. It could solve a symmetric positive-definite system of linear equations of dimension N in exactly N steps. As many practical problems at that time belonged to this category, CG algorithm became rapidly popular. It remains popular even today due to its immense computational power. But despite its amazing computational ability, mathematics of this algorithm is not easy to learn. Lengthy derivations, redundant notations, and over-emphasis on formal presentation make it much difficult for a beginner to master this algorithm. This paper aims to serve as a starting point for such readers. It provides a curt, easy-to-follow but minimalist derivation of the algorithm by keeping the sufficient steps only, maintaining a uniform notation, and focusing entirely on the ease of reader.