() literally means , waste, trash, litter, garbage, rubbish... and it's often used like "crap" or "a piece of crap". I think it means something like []{} or []{} (worthless, good-for-nothing..?).
The referee
the case particle as a subject marker
crap, rubbish
The auxiliary verb
only, It's only~~
So I think it's like... "It's only he's a crap referee!" "It's only because the referee is rubbish!"