The material itself is not so important, I think, but for sure the alloy jockeys last longer than plastic ones.
Besides that, the main adventage comes with the (often coming together with alloy pulleys) BEARING you already mentioned. Besides eliminating lateral play and so improving shifting precision (in my perception), they also decrease rolling resistance and improve drivetrain efficiency. Finally, these bearings ~~work better~~ suffer less without lube (don't wear out inside like plastic pulleys), and require much less lubing in the first place.
Well, these are my insights. Hope it helps.