You never apply Pollard $\rho$ to a number unless you have already proved that the number is composite. But there are ways to prove a number is composite that are far, far faster than looking for a prime factor of the number. Look up "Fermat's Little Theorem".