There is more to it!
Rejection sampling gives you independent samples from the exact distribution. MCMC gives you correlated samples that are not from the exact distribution.
This sounds like MCMC only has disadvantages. However, the correlation between successive samples decays, so you may simply skip some samples after each draw, and the distribution of your samples approaches the correct one exponentially fast (with respect to the number of samples you generate)
Finally, MCMC has the advantage that you can use it even when you the density only up to a multiplicative constant, which is the case in most Bayesian computations.
Furthermore, rejection sampling of course can take arbitrarily long for the generation of a single sample, which is another reason for which you might want to prefer to live with the imperfections of MCMC.