Fit is the most important thing I look for. A cycling top should fit snugly, without being skin-tight. Having less extra fabric will mean less air resistance, which makes a surprising difference at higher speeds. Also, too much excess fabric will flap in the wind, which can be annoying too.
Make sure that the length in back is long enough to cover your lower back when you're riding in an aggressive position. If a long-sleeve, the sleeves need to be long enough to cover your wrists with your arms extended out front. (some normal clothing fails these two checks)
One last caveat about jersey style: make sure the zipper opens far enough for you to be able to take it off after a long sweaty ride. I have one with a short zipper and it's a real struggle.
For materials, there's lots of nice synthetics to choose from that are lightweight, wick away moisture and are very comfortable in hot weather. But for cool or wet conditions, nothing beats wool.