In general, if $|G:H|$ is finite, then the core $K$ of $H$ in $G$, which is the intersection of the conjugates of $H$ in $G$, satisfies $K \le H \le G$ with $K \unlhd G$ and $G/K$ finite. Using that, it is clear that virtually-poly-abelian (same as virtually solvable) implies poly-virtually-abelian.
The converse is also true. By using induction on the length of the poly-virtually-abelian series of $G$ we can reduce to the situation where $G$ has a series $1 < H \lhd K \lhd G$ with $H$ and $G/K$ solvable and $K/H$ finite. We may assume that $H$ is the largest normal solvable subgroup of $K$, and so it is characteristic in $K$ and hence normal in $G$. Let $H < C$ with $C/H = C_{G/H}(K/H)$. Then $K/H$ finite implies that $G/C$ is finite. Also $(C \cap K)/H = Z(K/H)$ is abelian and $C/(C \cap K) \cong CK/K$ is solvable, so $C$ is solvable. Hence $G$ is virtually solvable and we are done.