The problem is $$ \min_{x\in\mathbb{R}^n} f(x) = \min_{x\in\mathbb{R}^n} \frac{1}{2} x^T A x + r^T x,$$ which has gradient $$ \
abla f(x) = A x + r.$$ The necessary condition $\
abla f(x_*)=0$ gives us that $Ax_*=-r$, which has a unique solution. Since the Hessian $A$ is positive definite we are guaranteed the existence and uniqueness of the solution to $A x_* = -r$, and that the point $x_*$ is a minimizer.
In the latter part of your question, do you mean perpendicular in $\mathbb{R}^n$ or perpendicular w.r.t. your inner product, i.e., $