What's the "ridge" in Ridge Regression? In normal least squares, we try to find $\hat\beta$ which minimizes $$\|y-X\beta\|^2$$ Ridge regression expands this to "penalize" certain values of $\beta$ via a matrix $\Gamma$: $$\|y-X\beta\|^2+\|\Gamma\beta\|^2$$ I'm wondering where the term "ridge" comes from. My best guess is that it has something to do with a geometric interpretation of the term $\Gamma$, but I can't find anything written about this anywhere.

As you can see in this link (page 5, col.2), Hoerl (presumably the inventor of ridge regression)

" _gave the name "ridge regression" to his procedure because of the similarity of its mathematics to methods he used earlier i.e., "ridge analysis," for graphically depicting the characteristics of second order response surface equations in many predictor variables._ "

In this link you can also find the name of the original paper of Hoerl [9]. So, in my interpretation the name is more due to the similarity of the method to earlier work of it's inventor rather due to it's characteristics.