AFAIK, there is actually nothing that explicitly says that he can fly, even when he uses his chariot. _Þrymskviða_ , verse 21, describes how he goes to Jotunheim:
> Then home the goats | to the hall were driven,
> They wrenched at the halters, | swift were they to run;
> The mountains burst, | earth burned with fire,
> And Othin's son | sought Jotunheim.
>
This does not suggest flying. Furthermore, in Hárbarðsljóð, Thor is returning from Jotunheim in the east, and gets involved in a flyting with a ferryman who refuses to carry him across. It is strange to imagine why he would have to do so if he could have taken his goats instead.
The only thing that comes close is how he is said to cause thunder, but there is no absolute need for that to come from free flight.