Thor's "ride" fits with his role as god of the common people. While most of the gods ride horses, Thor drives a wagon, or walks. (He walks across Bifrost, the rainbow bridge, according to the _Prose Edda_.)
As for the goats, I'm just guessing here, but there's a verse in the poem _Havamal_ , which is supposed to be the widsom of Odin himself:
> One's own house is best, though small it may be;
> each man is master at
> home; though he have but two goats and a bark-thatched hut
> 'tis better than craving a boon.
So a goat would be the least that a subsistence farmer could get by with - but Thor's goats are special, because he can eat them and use them for draft animals time and again. Also, anyone who can yoke a goat and make it pull a wagon is clearly possessed of supernatural powers.