Explain the architecture of a Deep Convolutional GAN (DCGAN) and distinguish it from a fully connected vanilla GAN. Discuss convolution and transposed-convolution layers, normalization, activation functions, and architectural guidelines for stable image generation.
What's actually being asked
Describe DCGAN's architecture — conv / transposed-conv layers, normalization, activations, design guidelines — and contrast it point by point with a fully-connected vanilla GAN.
Answer
Vanilla GAN (fully connected)
G: noise z → stack of Linear (fully connected) layers → reshaped into an image at the output.
D: flatten the image into a vector → stack of Linear layers → single Sigmoid output.
Treats the image as a flat vector and ignores spatial structure entirely — a pixel next to another pixel is treated no differently than two unrelated pixels. Works for tiny, low-resolution data but scales badly and tends to produce blurry, low-detail images as resolution grows.
DCGAN (Deep Convolutional GAN)
Replaces the fully connected stacks with convolutional structure that respects spatial locality in images.
Generator: noise z (reshaped into a small spatial tensor) → a stack of transposed convolution (fractionally-strided convolution) layers, each progressively increasing spatial resolution while reducing channel depth → output layer is a transposed conv producing the final image, with Tanh activation (data scaled to [-1,1]).
Discriminator: image → a stack of strided convolution layers, each progressively decreasing spatial resolution while increasing channel depth → final layer flattens to a single scalar, with Sigmoid activation.
Normalization
Batch Normalization is applied after most conv/transposed-conv layers, in both G and D — except on G's output layer and D's input layer, where it would wash out the sample-specific statistics that matter most. BatchNorm stabilizes training by keeping activation statistics consistent through a deep stack, which helps prevent some of the gradient-flow problems that otherwise push training toward mode collapse.
Activation functions
G hidden layers: ReLU.
G output layer: Tanh.
D hidden layers: LeakyReLU (small negative slope, e.g. 0.2) — avoids dead units and keeps gradient alive on both sides, which matters because D needs to stay sensitive to subtle real-vs-fake texture differences.
D output layer: Sigmoid.
Architectural guidelines (the DCGAN recipe)
Replace pooling layers with strided convolutions in D and transposed convolutions in G — lets the network learn its own up/down-sampling instead of a fixed pooling rule.
Use BatchNorm in both G and D, but not on G's output layer or D's input layer.
Remove fully connected hidden layers for deeper architectures — go fully convolutional.
Use ReLU in G for every layer except the output (Tanh); use LeakyReLU in D for every layer.
Diagram
See the reference diagram — note the symmetry: G's transposed-conv stack upsamples z into an image, D's conv stack downsamples an image into a score; the two are near mirror images of each other.
Why this matters
Convolutions share weights across spatial locations and exploit locality — a 3×3 kernel learns a reusable local pattern detector instead of a separate weight for every pixel pair, the way a flattened Linear layer would. That is exactly why DCGAN produces noticeably sharper, more spatially coherent images than a vanilla fully-connected GAN at a comparable parameter budget, and why it scales to realistic image resolutions where vanilla GANs tend to fall apart.
Exam tip
If the question says "distinguish from vanilla GAN," open with the one-line structural difference — conv/transposed-conv layers replacing Linear layers — before listing the guidelines. That's the core distinguishing fact graders look for first; the guideline list is supporting detail.