Explain the architectural design of the Generator and Discriminator in a GAN. Illustrate their input and output layers, activation functions, and information flow during one training iteration.
What's actually being asked
Describe how G and D are built layer by layer — inputs, outputs, activations — and trace exactly what happens to the data and the gradients during a single training iteration.
Answer
Generator G
Input: a random noise vector z ∼ p_z(z), usually z ∼ N(0, I), dimension e.g. 100.
Body: a stack of fully-connected layers (vanilla GAN) or transposed-convolution layers (DCGAN-style) that progressively upsample z towards the shape of real data.
Hidden activations: ReLU or LeakyReLU — keeps gradients flowing through the stack without saturating too early.
Output layer: shape matches real data; activation is Tanh if real data is scaled to [-1,1], or Sigmoid if scaled to [0,1].
Output: G(z) — a fake sample living in data space.
Discriminator D
Input: a sample x, either a real one (from p_data) or a fake one (G(z)).
Body: fully-connected layers (vanilla) or convolution layers (DCGAN) that progressively compress the input down to a single scalar.
Hidden activations: LeakyReLU is preferred over plain ReLU — avoids dead units and keeps gradient alive on both sides of the activation, important since D has to stay sensitive to subtle real-vs-fake differences.
Output layer: a single neuron with Sigmoid activation → D(x) ∈ (0,1), interpreted as the estimated probability that x is real.
Diagram
See the reference diagram: noise flows through G into a fake sample; both the fake sample and a real sample flow into the same D instance, which outputs one real/fake score.
Information flow during one training iteration
Sample a batch of real data x ∼ p_data and a batch of noise z ∼ p_z.
Forward z through G → fake batch G(z).
Forward both the real batch and the fake batch through D → D(x) and D(G(z)).
Compute D's loss, −[log D(x) + log(1 − D(G(z)))]; backpropagate; update D's weights only (G is frozen for this step).
Sample a fresh noise batch z, forward through G, then through the just-updated D again to get D(G(z)).
Compute G's loss (non-saturating form used in practice), −log D(G(z)); backpropagate the gradient through D's weights into G, but update G's weights only (D is frozen for this step).
Repeat steps 1–6 for many iterations.
Yaha samajhna zaroori hai ki — G's gradient must pass through D's forward computation (chain rule), but only G's parameters get updated in step 6. D is reused as a fixed function during that step, not retrained.
Exam tip
The diagram carries real marks here — always show noise as G's only input, one shared D instance taking both real and fake samples, and state explicitly that G's weight update requires a forward (and backward) pass through D while D's own weights stay frozen during that step. Examiners frequently probe this freeze/update split.