Naive Bayes vs logistic regression on binary classification

What's actually being asked

Take one concrete binary classification setup and contrast Naïve Bayes (generative) against Logistic Regression (discriminative) across assumptions, training, decision-making, advantages, limitations.

Answer

Setup

Binary classes y ∈ {0, 1} (e.g., "disease present" vs. "disease absent"), with feature vector x (e.g., test results). Naïve Bayes and Logistic Regression are the textbook generative/discriminative pair because they can even converge to the same linear decision boundary under the right conditions, while getting there completely differently.

How each approaches the problem

Naïve Bayes (generative): models P(x,y) = P(y)·P(x|y), assuming features in x are conditionally independent given the class: P(x|y) = Π_i P(x_i|y). Classification applies Bayes' theorem: P(y|x) ∝ P(y) Π_i P(x_i|y), picking the class with the higher posterior.

Logistic Regression (discriminative): models P(y=1|x) = σ(w·x + b) = 1/(1+e^{-(w·x+b)}) directly, with no assumption about how x is distributed within each class.

Comparison

Aspect	Naïve Bayes (generative)	Logistic Regression (discriminative)
Assumptions	Features are conditionally independent given the class; assumes a specific form for P(x|y) (e.g., Gaussian or categorical per feature)	No assumption about P(x)'s shape; only assumes the log-odds of y is a linear function of x
Training procedure	Closed-form: estimate P(y) as class frequency, estimate each P(x_i|y) from per-class feature statistics — no iterative optimization needed	Iterative: minimize cross-entropy loss over the training set via gradient descent to fit w, b
Decision-making	Compute P(y|x) indirectly via Bayes' theorem from the estimated prior and likelihoods, compare across classes	Compute P(y|x) directly from the learned linear function passed through a sigmoid, threshold at 0.5
Advantages	Works well with small/limited labeled data; fast, closed-form training; naturally extends to more than two classes and handles missing features via marginalization	Usually more accurate when the independence assumption is clearly false; directly optimizes the quantity being evaluated, so tends to be better calibrated with enough data
Limitations	Accuracy degrades when features are correlated (independence assumption violated) — common in real data	Needs more labeled data to fit well; provides no way to generate new samples or reason about P(x); purely a decision tool

The unifying insight

Both end up computing something that looks like a linear decision boundary in x under common assumptions (e.g., Gaussian likelihoods with shared covariance for Naïve Bayes), but Naïve Bayes gets there by first modeling the generative process per class, while Logistic Regression gets there by directly fitting the boundary — this is the clearest concrete illustration of "same destination, different road" between the two paradigms.

Exam tip

If the question gives you a dataset, actually plug in the Naïve Bayes formula P(y|x) ∝ P(y) Π_i P(x_i|y) with the given/illustrative numbers rather than stopping at the formula — numeric questions in this topic almost always want the posterior computed out.