Extremely wealthy (or powerful) people can get away with offending against conventions that other people are bound by, because other people don't want to offend them. Also, an ordinary person wanted to do business with the collector would have had to show that he was taking the matter seriously by dressing properly, but either a duke or a millionaire could show that just by showing up to make the deal personally.