Let the height of the rectangle be $x$ and the width $y$. Then
* the upper side of the area $4$ triangle is $8/x$;
* the upper side of the area $3$ triangle is $y-(8/x)$;
* the left side of the area $3$ triangle is $\dfrac{6}{y-(8/x)}$.
So by looking at the triangle of area $5$ we have $$5=\frac12\,y\,\left(x-\frac{6}{y-(8/x)}\right)$$ which simplifies to $$(xy)^2-24(xy)+80=0$$ with solutions $xy=20,4$. Obviously $xy>12$ so reject $4$, so the rectangle has area $20$ and the triangle has area $8$.