? is not necessarily rude, but it is certainly informal. As such, it should probably only be used with friends or family or in an informal environment. Using it outside those boundaries might risk it sounding somewhat brusque or perhaps even rude.
A standard polite alternative is:
> What is that?