Sometimes the `vi` command is an alias for `vim` and when called as `vi` enables its vi-mode.
Even in traditional mode backspace is deleting the character, but does not display it as deleted immediately. (After pressing ESC the characters are gone.)
Guess you have to choose between using `vi` which comes with the described behavior or using `vim` which is able to do it the way you expect it.