This is Feferman's extension of $\sf ZFC$, and it is a conservative extension of $\sf ZFC$. But note that you cannot state that $E$ is an elementary substructure of the universe with an axiom, since that would violate Tarski's theorem. You can, however, do it one axiom at a time.
To see that the new theory is a conservative extension, note that it follows immediately from the reflection theorem: everything true in $V$, is true in a countable transitive model (well, here we apply Lowenheim-Skolem and Mostowski's collapse lemma).
I couldn't find a specific citation, but I did find several references by other people to the following paper.
> Feferman, Solomon. "Set-Theoretical foundations of category theory", Reports of the Midwest Category Seminar III, Lecture Notes in Mathematics Volume 106, 1969, pp 201-247.