Your speculations in the question about what the contrast might mean are on point. Specifically, the poem moves from grandiose claims to mundane reality. The claims are about the past ( _recall_ ) or the future ( _next thousand years_ ); the reality is the present ( _am waiting_ ). That the day is _windy_ underlines the contrast between reality (cold wind) and the claims (hot air). Or perhaps there's a continuity rather than a contrast between the bluster of the time travel claims and the windiness of the day.
The speaker's waiting for a _bus_ also brings the claims of traveling through time with traveling through space. While others claim to have mastered time travel, the speaker is trying merely to get from one place to another, and hasn't even begun the bus journey.