Let your total population be $T = M + W$ where $M$ is the number of men and $W$ is the number of women.
Then there are $\binom{T}{20}$ ways of choosing $20$ people. If you want to know the probability of selecting $m \le \min\\{20, M\\}$ men and $w \le \min\\{20,W\\}$ women (where $m+w = 20$), then you have $\binom{M}{m}$ and $\binom{W}{w}$ ways of selecting the men and women, respectively. Therefore, your probability is
$$\frac{\binom{M}{m}\binom{W}{w}}{\binom{T}{20}} = \frac{\binom{M}{m}\binom{W}{w}}{\binom{M+W}{m+w}}$$