If you take a number and you first add the 20% and then take the 20% you don't obtain the original number. In fact $100 + 20\% = 120$ and $120 - 20\% = 96$.
So, since VAT is "adding $20\%$" the inverse is not "subtract $20\%$".
Since "adding $20\%$" is equivalent to multiply by $1.2$ then dividing by the same quantity is correct.
If you want to subtract a percentage then the right percentage is $$ 100 \cdot \left( 1- \frac{1}{1.2} \right) = 100 \cdot 1.\bar 6 = 16.\bar 6 \% \approx 16.67\% $$