(1) Let $s_3=s_2s_1r$. Then $s_1,s_3\in g_1$ and $s_2\in g_2$, so $s_1^{-1}\in g_1$ and $s_2^{-1}\in g_2$. Finally, $r=s_1^{-1}s_2^{-1}s_3$ is then in the group generated by $g_1$ and $g_2$, since that group certainly contains all products of elements of $g_1$ and $g_2$.
(3) $Sp(1)$ is homeomorphic to $S^3$ (see the bottom of p. $36$), so it’s compact. $A$ is the union of two open sets, so it’s open. $Sp(1)\setminus A$ is therefore a closed subset of the compact set $Sp(1)$, so it must be compact.
(4) He’s looking at the family
$$\\{\theta(U)\cap A:U\text{ is a compact neighborhood of the unit matrix in }SO(3)\\}\;;$$
Letting $U$ run over all these compact neighborhoods is just letting it become each of them in turn, as a dummy variable.