I don't have a good reference on perturbations of PF eigenvalues but it is fairly simple to make examples. For $\epsilon\geq 0$: $$ M_\epsilon = \left( \begin{matrix} 1 & 1 & 0 \\\ 0 & 1 & 1\\\ \epsilon & 0 & 1\end{matrix}\right) $$ has char pol $p(\lambda)=(\lambda-1)^3-\epsilon\ $ so the leading eval is $\lambda=1+\epsilon^{1/3}$, i.e. is $\frac{1}{3}$-Hölder continuous in $\epsilon$.
In dimension $d$ you may at worst get $\frac{1}{d}$-Hölder.
It can not get worse simply because you look at zeroes of a polynomial with coefficients that are polynomials in matrix elements. A root of order $d$ behaves $1/d$-Hölder continuously in the matrix elements and an isolated zero depends analytically upon the matrix.
In particular, from this you also see that in the strictly positive case, the leading eval is isolated (PF-theorem) so behaves analytically with the matrix.