Your first idea is good. If there are $3x$ people who ate ice cream and $2x$ who ate cake, then $x$ is at most $16$, since $3x\leq 50$, the total number of people. So $2x$ is at most $32$. The best case scenario is obviously when all of those who ate cake also ate ice cream, in which case we would have $32$, which is the answer.
Your approach is wrong because $(3x)+(2x)-50$ is not, in fact, what you're trying to maximise. You're taking $$\text{ice-cream eaters}+\text{cake eaters}-\text{total},$$ which does not in fact correspond to the number of people who ate both, as you have implicitly assumed that no one ate neither. You can draw out a Venn Diagram to convince yourself of this.