I admit, I like to use vi but I like it derpy. I tend to use arrow keys a lot anyway and expect backspace to work the same everywhere. So, slight change required some times.

In your home directory, you may need to create the file...

$ cat .vimrc 
set nocompatible
set backspace=2