Your approach is correct, but the problem does not require that the two functions are normal in the crossing point. We have only to determine for which values of $a$ there exists a point where the slope of the curve $y=4/x$ is normal to that of the line $y=\frac{a-3}{a}x-\frac{a^2-1}{a}$. Thus we can focus on the first of the two equations correctly shown in the question, i.e.
$$\frac{a-3}{a}=\frac{x^2}{4}$$
from which
$$x=\sqrt{\frac{4(a-3)}{a}}$$
The last equation has real solutions only for $\frac{a-3}{a}\geq 0$, so that we get $a< 0$ or $a>3$. Note that the values $a=3$ and $a=0$ have to be excluded because the slope of the line reduces to $0$ and $\infty$, respectively.