Fetal testis produces testosterone from cholesterol. There is a peak of production around 15 weeks of gestation (the "masculinization programming window"). So the genotype of the fetus can affect testosterone levels directly via effects on the biosynthesis of the hormone, or indirectly by defective regulation of the pathway's activity.
However, exposure to chemicals in the mother's environment is also thought to play a part in determining testosterone levels. The classic example of this is the phthalate esters used as plasticizers.
> Scott et al. (2009) Steroidogenesis in the Fetal Testis and Its Susceptibility to Disruption by Exogenous Compounds. Endocrine Reviews 30: preprint freely available here.