The spicy feeling essentially is the feeling of heat and pain. It is caused by the molecule Capsaicin (< binding to the ion channel TRPV1 (< activating it.
You can temporarily get rid of the pain by cooling (as you would with any other injury) - hence the effect of ice.
Unfortunately, this does not get rid of the cause of the pain, i.e. the molecule bound to the ion channel. Capsaicin is not very water soluble, but only fat soluble. This is why drinking milk helps: it contains microdroplets of fat, which can solvate the Capsaicin, which then can get washed away from the "pain receptor".