Are Shimano jockey wheels compatible between groupsets, provided same speeds? Recently my Shimano 105 (10 speed, RD-5700) jockey wheels have started squeaking during shifting, so they'll be getting replaced. I'm thinking about moving up to Ultegra jockey wheels for the ceramic bearings/bushings, but I don't know if they're compatible. What are the general factors that define compatibility? Is it more than just tooth count? Anything specific to keep an eye out for?

They are often compatible between different companies. Certainly between different groupsets of the same companies. The manufacturer normally states where they can be used and often that is multiple systems (like both Shimano and Campagnolo, some provided spacers may need to be used and may only work for some range of speed counts). Of course, if you buy a Shimano product, they will tell you it is for Shimano only. 10 speed Shimano should be compatible with other 10 speed Shimano.

But before replacing the wheels, I would try to service the current ones. I think 105 generally has pulley wheels with bushings. They can be easily services and likely need that more often than pulley wheels with bearings. How to do that is in the video below

You can of course consider ceramic bearings but the jump from bushings to any sealed bearings is likely more important than the jump from metalic bearings to ceramic bearings.