The Wikipedia entry, and a few other places that mention this, give as reference the popular book of John Derbyshire on the Riemann hypothesis. The book itself mentions this in the spirit of an apocryphal tale, with the passing remark that "I wouldn't put it past him", and gives no reference at all.
It's probably best to treat this story with utmost suspicion until better evidence turns up. This is the kind of tale that proliferates in mathematics because on the one hand it adds to the legend of the people we admire, whom we suspend critical judgement for in favour of awe and wonder, and on the other hand it plays on the insecurities of the average aspiring mathematician.
For people with such pretense of belief in reason, mathematicians are a little too often purveyors of rumours and hearsay themselves.