If each prisoner individually picks an arbitrary representative of the equivalence class of the actual sequence of hat colors, and guesses their own hat color according to this arbitrarily selected one, then it is possible that _every single prisoner_ guesses their hat color _wrong_. There is no limit.
They need to coordinate and agree on one representative for every single equivalence class and stick to the chosen representative, because then only finitely many of them can be wrong instead of arbitrarily many of them.