Diagnosing mode collapse in a digit GAN

What's actually being asked

Given the described symptoms, name the exact failure mode, explain why it happens, and prescribe specific fixes with the mechanism each one targets.

Answer

Identifying the problem

This is mode collapse, not vanishing gradients. The giveaway is the combination of symptoms: G is still producing samples confident enough to sometimes fool D, but only from a narrow subset of digit styles — that is a diversity failure, not a "G has stopped learning" failure. If this were vanishing gradients instead, G's output quality would stall or degrade broadly, without the specific signature of "only a few styles, repeated."

Here, D becoming highly confident is a consequence of mode collapse, not an independent problem: once G narrows down to a handful of digit styles, D's job becomes easy — recognizing a few repeated patterns as fake is a much simpler classification task than distinguishing the full diversity of real handwriting — so D(x) → 1 and D(G(z)) → 0 sharply. D's confidence is a symptom of how easy G made the job, not a separate vanishing-gradient issue.

Why it happens

• G's objective only needs to fool the current D; the cheapest way to do that is often a few modes that already slip past D, rather than covering the whole distribution.
• The standard GAN loss has no explicit term rewarding variety across generated samples — minimizing −log D(G(z)) per sample says nothing about how different one sample is from another.
• Once G settles on such modes, G's gradient signal comes only from D's current decision boundary, which may not "see" the unexplored regions of style-space at all, so G has little pressure to move away.

Recommended fixes, with mechanism

1. Minibatch discrimination — feed D statistics across the whole minibatch (e.g. pairwise feature distances between samples in the batch), not just one sample at a time, so D can directly flag and penalize batches that look too similar to each other. This gives G a diversity-aware gradient — it can no longer win by submitting near-identical digits within a batch.
2. Unrolled GANs — let G's update account for several lookahead steps of how D will respond, instead of reacting to a single frozen D snapshot. This removes the incentive to exploit a blind spot in D that is predicted to close within those unrolled steps.
3. Switch to a Wasserstein-based loss (WGAN / WGAN-GP) — the Wasserstein distance is a smoother, everywhere-informative metric between distributions, unlike the JS-divergence-based standard GAN loss, which behaves poorly when p_g and p_data barely overlap — exactly the regime created by mode collapse. A smoother loss landscape reduces the incentive for G to get stuck exploiting one narrow mode.
4. Diversity-promoting regularizers — e.g. a feature-matching loss, or an explicit term rewarding variety across a batch of G's outputs — directly attacks the root cause (no diversity term in the vanilla loss) instead of relying on architectural tricks alone.

Simple shabdo mein: pehle yeh pehchaano ki G ne sirf chhoti si range exploit kar li hai, toh fix bhi usi diversity-gap ko directly target karna chahiye — architecture replace karna zaroori nahi, loss/training-signal mein diversity laana zaroori hai.

Exam tip

Explicitly write the sentence distinguishing mode collapse from vanishing gradients — this scenario is deliberately constructed so the two can be confused. Examiners are checking whether you understand that "D is confident" can be a symptom of mode collapse rather than proof of a separate vanishing-gradient problem.