What is the interpretation of the expectation notation in the GAN formulation? I'm confused about the expectation notation in the context of GAN loss functions. The GAN loss for the discriminator is binary cross-entropy. ie: is this real or not. real = $D(x)$ (ie: give the discriminator a real image) fake = $D(G(z))$ (ie: generate a fake image and ask discriminator what it is) Then the binary crossentropy is: $$log(p) - log (1-p)$$ When used as a GAN loss we replace p with either "real class" or "fake class". $$log(real) - log (1-fake)=\\\ log(D(x)) - log (1-D(G(z)))$$ So far, this is ok (i think haha). But the actual formulation adds an expectation sign... which I don't understand why it's there. $$E_{x~data}log(D(x)) - E_z log (1-D(G(z)))$$

It comes from the fact that $D$ wants to maximize the loss over all potential examples it sees and not just a single example: for real data points, this is done by minimizing the expectation over all true data points of $\log(D(x))$ (your first term). For false points (which are generated by $G$ from the noise distribution $z$), it wants to maximize the expectation over all of these points of $\log(1-D(G(Z)))$, which is the second term.