There are a few posts asking this question. Here is my answer:
We only need to show that after eliminating $a_{2,1}$, diagonal dominance is preserved, i.e., $$ \left|a_{2,2}-a_{1,2}{a_{2,1}\over a_{1,1}}\right|>\sum_{i=3}^n\left|a_{2,i}-a_{1,i}{a_{2,1}\over a_{1,1}}\right|, $$ which is equivalent to $$ |a_{2,2}a_{1,1}-a_{1,2}a_{2,1}|>\sum_{i=3}^n|a_{2,i}a_{1,1}-a_{1,i}a_{2,1}|. $$ But this is true: \begin{eqnarray*} \sum_{i=3}^n|a_{2,i}a_{1,1}-a_{1,i}a_{2,1}|&\le& |a_{1,1}|\sum_{i=3}^n|a_{2,i}|+|a_{2,1}|\sum_{i=3}^n|a_{1,i}| \\\ &<& |a_{1,1}|(|a_{2,2}|-|a_{2,1}|)+|a_{2,1}|(|a_{1,1}|-|a_{1,2}|) \\\ &=&|a_{1,1}||a_{2,2}|-|a_{2,1}||a_{1,2}|\\\ &\le& |a_{1,1}a_{2,2}-a_{2,1}a_{1,2}| \end{eqnarray*}