It is definitely literal. It's possible that it may be metaphorical as well, although I don't see what the metaphor could be.
When _the Great Gatsby_ was written, everybody smoked. The air at parties would gradually get smokier and smokier. Towards the beginning of the party scene (a page earlier than the quote in the OP) Nick goes out to get cigarettes:
> then there were no cigarettes, and I went out to buy some at the drugstore on the corner.
So if everybody ran through all their cigarettes, it's clear that people are smoking heavily.