Relaxation Methods:
   change xi so that ri  →  0
1.5
   xi +1  = xi + ∆ xi where
   ∆ xi = λi pi , i = 0,1,2,3,......,m
   λi suitably chosen coefficient,
   pi search direction
if xn= x  direct methods,
else m = ∞
In the relaxation methods the residual vectors play an important role since they are used for finding better approximations. This notion was introduced by Sir Richard Southwell, a British engineering professor of the university of Oxford who invented the point relaxation method. He harbored a rather shortsighted view about systematic relaxation methods adapted to computers: (citations from articles by D. Young and Garrett Birkhoff in “History of Scientific Computing”)
,,Any attempt to mechanize relaxation methods would be a waste of time.“ He did not believe that Computers could compete with human intuition and ingenuity in applying relaxation methods. He emphasized ,,the freedom left to the [human] computer, to decide the nature of his next step“ in relaxation methods.”

L. Fox‘s described Southwell‘s relaxation method:
The success of the method (and it was successful even with the meager computing equipment then available) depended significantly on the ability of the human eye and brain to pick out quickly the largest of a sequence of numbers or a cluster of such numbers, to recognize patterns of numbers, and to forecast the overall effects of relaxation operations. In fact, it was rather like a game of chess.