Unless I'm missing something, both of the things you suggest are true. This is because the division algorithm works in $R[X]$ for any ring $R$, provided that the lead coefficient of the divisor is a unit.
$1$ is certainly a unit, so we can always divide by a monic polynomial. Thus for any $a \in R$, $f\in R[X]$, $f(X)=(X-a)g(X)+c$ for some constant $c$. Thus if $f(a)=0$, $c = 0$