Designing a multi-class image classifier: generative vs discriminative

What's actually being asked

Design (at a conceptual level) a multi-class image classification pipeline built both ways — generative and discriminative — compare them on five named axes, and justify a practical final choice.

Answer

System design

Both branches share a feature-extraction front end (raw pixels are rarely classified directly; extract/learn a feature vector x per image, e.g., via a CNN backbone or simpler hand-engineered features), then diverge into the generative and discriminative paths shown in the diagram below — each path reaching a final class prediction through a different route.

Generative branch: fit a Gaussian Mixture Model (or Naïve Bayes) per class on the extracted features — estimate P(y) from class frequencies and P(x|y) as a Gaussian mixture over each class's feature distribution. Classify a new image by computing P(y|x) ∝ P(x|y)P(y) for every class and picking the maximum.

Discriminative branch: train a neural network (or SVM) end-to-end to directly output P(y|x) (softmax over classes for a NN) or a maximum-margin decision function (SVM), optimizing a classification loss directly against the labels.

Comparison

Axis	Generative (GMM / Naïve Bayes)	Discriminative (NN / SVM)
Probability estimation	Explicitly estimates P(x|y) and P(y), then derives P(y|x) — gives a full model of each class's feature distribution	Estimates P(y|x) directly (softmax probabilities for NN); SVM doesn't produce calibrated probabilities at all, just a margin-based decision
Decision boundaries	Boundary is implicit, shaped by the assumed form of each class's distribution (e.g., ellipsoidal for Gaussian components) — can be too rigid for complex real image classes	Boundary is learned directly and can be arbitrarily complex/non-linear (deep NN) or maximum-margin (SVM), typically fitting real class boundaries better
Handling unseen data	Can flag genuinely novel/out-of-distribution images naturally, since a very low P(x) under all classes is a usable anomaly signal	Usually forced to assign some class to any input (softmax always sums to 1) — poor native novelty detection without extra calibration
Training requirements	Can be fit with relatively little labeled data per class; training is largely statistical estimation, not iterative optimization	Needs substantially more labeled data (especially deep NNs) and iterative gradient-based training, often with data augmentation and regularization
Expected performance	Degrades if the Gaussian/independence assumptions don't match true image feature distributions (common with raw or shallow features)	Typically achieves higher raw classification accuracy on complex, high-dimensional image data when enough labeled data is available

Justification for a practical system

For a real multi-class image classifier with a reasonably sized labeled dataset, the discriminative approach (a neural network) is the practical choice — image feature distributions are rarely well-approximated by Gaussians or independent features, so a NN's directly-learned, flexible decision boundary wins on raw accuracy. The generative approach (GMM/Naïve Bayes) remains valuable as a lightweight baseline and specifically for out-of-distribution/novelty detection as a complement to the NN, since it's the one piece the discriminative model can't do natively — a hybrid system (NN for classification, GMM for flagging "this doesn't look like any training class") is often the best of both.

Exam tip

"Justify your choice" questions want a reasoned trade-off, not just "neural networks are better." Explicitly naming what the generative side gives up (accuracy on complex data) and what it uniquely keeps (novelty detection, lower data needs) earns the extra marks.