HINT: For any $p\in\Bbb R^2$ and any $\epsilon>0$, the set $\Bbb R^2\setminus\operatorname{cl}B(p,\epsilon)$ is open. (Here $B(p,\epsilon)$ is the open ball of radius $\epsilon$ and centre $p$.)
**Added:** This doesn’t arise in your specific example, but in general you need more than that $p$ is in the boundary of the compact set: you need it to be a limit point. If your original compact set were the closed unit disk together with a point $p$ not contained in it, for example, $p$ would be in its boundary, but removing $p$ would still leave a compact set.