The number of ways of selecting a subset of size $k$ from a set of $n$ objects is given by the formula $$\binom{n}{k} = \frac{n!}{k!(n - k)!}$$ where $n!$, read "$n$ factorial," is the product of the first $n$ positive integers if $n$ is a positive integer and $0! = 1$. The notation $\binom{n}{k}$ is read "$n$ choose $k$."
There are $\binom{5}{3}$ ways to select a subset of three of the five angles.
Of the five angles, three are acute and two are not. If exactly two of the three selected angles are acute, one of the two other angles must be selected. Therefore, the number of favorable selections is $$\binom{3}{2}\binom{2}{1}$$
Hence, the probability that exactly two acute angles will be selected when three of the five angles are selected is $$\frac{\dbinom{3}{2}\dbinom{2}{1}}{\dbinom{5}{3}} = \frac{3 \cdot 2}{10} = \frac{3}{5}$$ as you found.