Mid-term final notes

Tip: Scoring tricks: Always draw a labelled diagram (even a simple box-arrow one). Write formulas in a box. Use a comparison table wherever the question says "compare / differentiate". Underline keywords: latent space, minimax, reparameterization, ELBO, mode collapse, self-attention, cross-attention, causal mask, chunking, top-k retrieval.

Tip: Highest-priority topics (by number of practice questions): GAN (architecture + loss + challenges) ≫ VAE (ELBO + reparam + KL) ≫ Generative vs Discriminative ≫ Diffusion (with numerical) ≫ Transformer + Self-attention numerical ≫ cGAN / StyleGAN ≫ RAG ≫ Prompting.

Generative AI is a class of AI models that learn the probability distribution of the training data, p(x), and then sample from it to create new data (images, text, audio, code) that looks like the training data but is not a copy.

Fundamental mechanism (how it works)

Large dataset → Learn patterns / distribution p(x) → Compress into latent space / parameters → Sample (noise z or prompt) → Decode → new data

1. Learning: model sees many examples and learns hidden structure (shapes, textures, grammar).
2. Latent representation: data is mapped to a compact space where similar things are close.
3. Sampling: pick a random point / give a prompt.
4. Generation: decoder / generator / denoiser turns it into a realistic output.

Characteristics

• Novelty – creates new samples, not copies.
• Probabilistic – same input can give different outputs (diversity).
• Learns from unlabeled data (mostly unsupervised / self-supervised).
• Controllable – conditioning with labels, text prompts, images.
• Multimodal – text, image, audio, video, code.
• Data-hungry and compute-heavy.

Types of generative models

Use cases (with example)

Major challenges: (1) Hallucination / factual errors, (2) Bias from training data, (3) Privacy (model may memorise patient data), (4) Copyright & deepfakes / misuse, (5) high compute cost & energy.

Why GenAI suits synthetic medical-image augmentation (Q1)

• Medical data is small, costly, privacy-restricted, and imbalanced (rare diseases).
• Generative model learns p(x) of scans → samples new, realistic, varied scans → bigger and balanced training set → better classifier, less overfitting.
• Synthetic data can be shared with less privacy risk (no real patient).
• Best choice: Diffusion (quality + diversity) or cGAN (class control); VAE if data is very small and stability matters.
• Must validate: radiologist review, FID score, and train-on-synthetic-test-on-real accuracy.

Joint vs conditional probability

Joint probability P(X, Y): probability that X and Y happen together. Example: P(email contains "free" AND is spam) = 0.12. Used by generative models. Applications: Naïve Bayes spam filter, HMM speech recognition, GMM clustering, image generation.

Conditional probability P(Y | X): probability of Y given X has happened. Example: P(spam | email contains "free") = 0.80. Used by discriminative models. Applications: Logistic regression credit scoring, SVM / CNN image classification, sentiment analysis.

Bayes' theorem link (derivation)

P(X, Y) = P(X | Y) · P(Y) = P(Y | X) · P(X)
 
⇒ P(Y | X) = P(X | Y) · P(Y) / P(X),  where P(X) = Σ_y P(X | y) P(y)
 
Posterior = Likelihood × Prior / Evidence

• Generative learns P(X|Y) and P(Y) (i.e. the joint P(X,Y)), then uses Bayes to get P(Y|X). It can also generate new X by sampling from P(X|Y).
• Discriminative learns P(Y|X) directly (or just a decision boundary). It cannot generate data.
• Decision rule for both: ŷ = argmax_y P(y|x). For generative: argmax_y P(x|y)P(y) (P(x) is same for all classes so ignored).

The accompanying illustration showed two classes side by side: generative models learn a region/likelihood (overlapping blobs) around each class — "what does each class look like?" (learns P(x|y)·P(y)) — while discriminative models just learn a single direct separating boundary line between the classes — "which side of the line?" (learns P(y|x)).

Comparison table (Q5, Q7)

Naïve Bayes vs Logistic Regression on the same binary problem (Q8)

Task: classify email as Spam (1) / Not spam (0) using word features x = (x₁ … xₙ).

Naïve Bayes (generative)

P(y|x) ∝ P(y) · Π_i P(x_i|y)

• Assumption: features independent given class.
• Training: count word frequencies per class (closed form, very fast).
• Decide: pick class with larger P(y)ΠP(x_i|y).
• • fast, works with small data. − independence assumption often false → poor calibration.

Logistic Regression (discriminative)

P(y=1|x) = σ(wᵀx + b) = 1 / (1 + e^(−(wᵀx+b)))

• Assumption: log-odds linear in x.
• Training: minimise binary cross-entropy with gradient descent.
• Decide: spam if σ(wᵀx+b) > 0.5 → linear boundary wᵀx+b=0.
• • higher accuracy with more data, handles correlated features. − needs labels, can't generate.

Tip: Famous result (Ng & Jordan): NB reaches its best accuracy faster (less data); LR reaches lower final error with enough data.

Multi-class image classification system design (Q9)

• Pipeline: images → preprocessing (resize, normalise) → feature extraction (CNN features / HOG) → classifier.
• Generative branch: GMM per class or Gaussian NB → P(x|class)·P(class) → argmax. Gives probability of the input itself, so it can flag unseen / out-of-distribution images.
• Discriminative branch: Softmax regression / SVM / CNN → learns boundaries directly → highest accuracy.
• Justified choice: for a practical system with enough labelled images, use a CNN (discriminative) for accuracy; add a generative model (or VAE reconstruction error) for OOD detection or data augmentation when data is small.

A VAE is a generative model with an encoder that maps an input x to a probability distribution (mean μ, variance σ²) in latent space, samples z from it, and a decoder that reconstructs x from z. Because the latent space is forced to be close to N(0, I), we can sample random z and decode it to generate new data.

Autoencoder vs VAE (Q10)

Why VAE is better for generation: the KL term makes the latent space continuous and centred, so any nearby point decodes to a valid image, and interpolation between two points gives smooth morphing.

Architecture (the diagram above)

Loss:

Loss = ‖x − x̂‖² (reconstruction) + KL(q(z|x) ‖ N(0,I))

Role of each part (Q11)

• Encoder q_φ(z|x): neural net (CNN/MLP) that outputs μ(x) and log σ²(x) — the "approximate posterior".
• Latent space: low-dimensional space (e.g. 2–128 dims) where each input is a Gaussian blob. Prior p(z) = N(0, I).
• Reparameterization trick: makes sampling differentiable (see below).
• Decoder p_θ(x|z): maps z back to data space; output layer sigmoid (pixels in [0,1]).
• Generation: throw away the encoder → sample z ~ N(0,I) → decoder → new image.

Reparameterization trick (Q13)

Problem: z ~ N(μ, σ²) is a random sampling step. You cannot backpropagate through "random sampling" — the gradient of a random draw with respect to μ, σ is undefined.

Solution: move the randomness outside the network into a separate variable ε.

z = μ + σ ⊙ ε,   ε ~ N(0, I),   σ = exp(½ · log σ²)
 
∂z/∂μ = 1,   ∂z/∂σ = ε   → gradients flow to the encoder

Significance: z still has distribution N(μ, σ²) (same randomness), but now z is a deterministic, differentiable function of μ and σ, so the whole VAE trains end-to-end with normal backprop / SGD. We predict log σ² (not σ) so the value can be any real number while σ itself stays positive.

ELBO derivation (Q14, Q15, Q17) — write all steps

Goal: maximise log-likelihood log p_θ(x) = log ∫ p_θ(x|z) p(z) dz. This integral is intractable (over all z), and the true posterior p(z|x) is also intractable. So we introduce an approximate posterior q_φ(z|x).

log p(x) = log ∫ p(x, z) dz
         = log ∫ q(z|x) · [ p(x, z) / q(z|x) ] dz
         = log E_q(z|x)[ p(x, z) / q(z|x) ]
         ≥ E_q(z|x)[ log p(x, z) − log q(z|x) ]      (Jensen's inequality: log E[·] ≥ E[log ·])
         = E_q[ log p(x|z) + log p(z) − log q(z|x) ]
         = E_q(z|x)[ log p(x|z) ] − KL( q(z|x) ‖ p(z) )   = ELBO

Exact relation (alternate derivation):

log p(x) = ELBO + KL( q(z|x) ‖ p(z|x) )   and since KL ≥ 0 ⇒ log p(x) ≥ ELBO

• Term 1 – reconstruction likelihood E_q[log p(x|z)]: how well the decoder rebuilds x. In practice = −BCE or −MSE.
• Term 2 – KL divergence KL(q(z|x)‖p(z)): how far the encoder's distribution is from the prior N(0,I).
• Maximising ELBO pushes up a lower bound on log p(x) (a better generative model) and makes q(z|x) close to the true posterior.
• VAE Loss = −ELBO = Reconstruction loss + KL loss.

KL divergence term (Q12)

KL(P‖Q) = Σ P(x) log [P(x)/Q(x)]   (≥ 0, = 0 only when P = Q, not symmetric)
 
For q = N(μ, σ²), p = N(0, 1) (closed form):
KL = −½ Σ_(j=1..d) ( 1 + log σ_j² − μ_j² − σ_j² )

Why needed / how it regularises:

• Without KL, the encoder would make σ → 0 and put codes far apart → becomes a plain autoencoder → latent space has holes, random z gives garbage.
• KL pulls every μ towards 0 and σ towards 1 → codes overlap and fill the space → continuous, complete latent space.
• Acts like a regulariser, prevents overfitting, enables sampling from N(0,I).
• Why both losses? Only reconstruction → good copies, no generation. Only KL → all codes identical, outputs blurry average. Together → balance between accuracy and smooth latent space. (β-VAE: Loss = Recon + β·KL; β>1 gives more disentangled features.)

Assumptions & limitations (Q15)

• Assumes prior N(0,I) and Gaussian (diagonal) posterior — too simple for complex data.
• Assumes pixel independence given z (Gaussian/Bernoulli likelihood).
• Limitation 1: Blurry outputs — MSE/BCE loss averages possibilities (GANs/diffusion are sharper).
• Limitation 2: Posterior collapse — with a powerful decoder, KL → 0 and the decoder ignores z.
• Limitation 3: ELBO is only a bound — there's a gap between ELBO and the true log p(x).
• Limitation 4: trade-off between reconstruction and KL (needs tuning β).
• Advantages: stable training, no mode collapse, meaningful latent space, encoder gives representations.

Design: VAE for small-dataset augmentation (Q16, Q18)

Small real dataset → Preprocess (resize, normalise [0,1]) → Train VAE (Recon + KL) → Sample z~N(0,I) or perturb real z → Decode synthetic images → Quality filter → Real + synthetic → train classifier → Test on REAL test set

• Class-wise generation: train one VAE per class or a Conditional VAE (add label to encoder & decoder).
• Latent sampling: random z ~ N(0,I), or encode a real image and add small noise, or interpolate between two images.
• Quality filtering: remove samples with low classifier confidence, high reconstruction error, or near-duplicates of training images (privacy).
• Evaluation: (1) Visual / expert check; (2) FID, IS scores; (3) Downstream test: train a classifier on real only vs real+synthetic, compare accuracy/F1 on the real test set — if accuracy improves, the synthetic data is useful; (4) TSTR (train on synthetic, test on real).

A GAN (Goodfellow, 2014) has two neural networks trained against each other: the Generator G turns random noise z into fake data G(z); the Discriminator D outputs the probability that its input is real. G tries to fool D, D tries to catch G. This competition (a two-player minimax game) makes G produce realistic data.

Analogy: G = counterfeiter making fake notes, D = police. Both keep improving until fakes are indistinguishable from real.

Architecture / workflow diagram (Q19, Q20, Q22, Q54)

Objective function (value function) — MUST memorise

min_G max_D V(D, G) = E_x∼p_data[ log D(x) ] + E_z∼p_z[ log(1 − D(G(z))) ]

• D maximises V: wants D(x) → 1 (log 1 = 0, max) for real, and D(G(z)) → 0 so log(1−0)=0.
• G minimises V: only second term depends on G; wants D(G(z)) → 1 so log(1−D(G(z))) → −∞.
• Optimal discriminator for fixed G: *D(x) = p_data(x) / (p_data(x) + p_g(x))**.
• Plugging D* back: V = −log 4 + 2·JSD(p_data ‖ p_g). Minimum when p_g = p_data → JSD = 0, D*(x) = ½ everywhere (Nash equilibrium, D just guesses).

Training algorithm (alternating updates) (Q21, Q24)

1. Sample mini-batch of m real images x and m noise vectors z.
2. Update D (G fixed) by gradient ascent: ∇_θd (1/m) Σ [ log D(x^i) + log(1 − D(G(z^i))) ]. Can repeat k steps.
3. Sample new noise z.
4. Update G (D fixed) by gradient descent on (1/m) Σ log(1 − D(G(z^i))) — in practice maximise log D(G(z)) (non-saturating).
5. Repeat until D(G(z)) ≈ 0.5 and samples look real.

How G learns gradually: G never sees real images. It only gets the gradient of D's output with respect to its fake image — this tells G "change these pixels in this direction to look more real". Early: blobs → mid: rough shapes → late: sharp, realistic samples.

Forward pass: z → G → G(z) → D → D(G(z)); x → D → D(x). Backward pass: D-loss gradients update only θ_d; G-loss gradients flow through D back into G, updating only θ_g.

Generator & Discriminator design (Q23, Q25)

Significance of latent input z: provides randomness → diversity (different z → different output); the latent space becomes organised (nearby z → similar images; directions = attributes like smile, age) → enables interpolation & editing. Without z, G would produce one fixed output.

GAN applications

Face synthesis (StyleGAN), Super-resolution (SRGAN), Image-to-image (Pix2Pix, CycleGAN), Colourisation, Data augmentation (medical, agriculture), Text-to-image (StackGAN), Inpainting, Tabular synthetic data (CTGAN), Anomaly detection.

Scenario designs (Q52, Q53, Q54)

• Diseased leaf generation (agriculture): collect few diseased-leaf images → resize 64×64/128×128, normalise [−1,1] → DCGAN or cGAN (label = disease type) → train with minimax loss → filter outputs (expert / classifier confidence) → mix with real → train disease classifier → measure accuracy gain + FID.
• Network-flow (tabular) GAN: numeric features → min-max / mode-specific normalisation; categorical (protocol, flag) → one-hot; G output: Tanh for numeric, Gumbel-Softmax for categorical; D = MLP (or LSTM/Transformer for sequences); use CTGAN / conditional vector for imbalanced classes, WGAN-GP for stability. Evaluate: per-column distribution similarity (KS test), correlation matrix difference, TSTR intrusion-detection accuracy, privacy (distance to closest record).
• Satellite images: DCGAN/Progressive GAN with conv layers, multi-band input (RGB + NIR), real path vs fake path into D, value function as above, WGAN-GP or spectral norm for stability; evaluate with FID and domain experts.

Standard losses (Q26, Q27)

Discriminator loss: L_D = −[ log D(x) + log(1 − D(G(z))) ]   (binary cross-entropy, real=1, fake=0)
 
Saturating (minimax) G loss: L_G = log(1 − D(G(z)))   (minimise)
 
Non-saturating G loss: L_G = −log D(G(z))   (minimise) ← used in practice

Tip: Remember: both G losses want D(G(z)) → 1. Only the shape of the curve differs. Non-saturating = "maximise the chance D is fooled" instead of "minimise the chance D is right".

Major training challenges (Q28)

1. Mode collapse: G produces only a few types of output (e.g. only digit "1") because those fool D. Low diversity. Cause: G finds one "safe" output; D adapts; G jumps to another mode — they chase each other.
2. Vanishing gradients: D becomes too good → D(G(z)) ≈ 0 → saturating loss gives no gradient → G stops learning. Also because JSD is constant (log 2) when distributions don't overlap.
3. Training instability / non-convergence: two networks optimise opposite objectives; losses oscillate, no clear stopping point; very sensitive to hyperparameters.
4. Others: no good evaluation metric (loss doesn't show quality → use FID/IS), checkerboard artifacts, overfitting/memorisation, hard to train at high resolution.

Fixes

LSGAN: L_D = ½E[(D(x) − 1)²] + ½E[D(G(z))²],   L_G = ½E[(D(G(z)) − 1)²]
 
WGAN: min_G max_‖D‖_L≤1 E[D(x)] − E[D(G(z))]
WGAN-GP penalty: λ · E[ (‖∇_x̂ D(x̂)‖₂ − 1)² ], x̂ = random mix of real and fake, λ = 10

Case: handwritten digits, only few styles, D very confident (Q29)

Diagnosis: Mode collapse + discriminator overpowering (vanishing gradient). Causes: D trains too fast/too strong, saturating loss, small batch, high learning rate, no diversity signal. Remedies: non-saturating or WGAN-GP loss, minibatch discrimination, label smoothing, add noise to D inputs, lower D lr / update G more often, spectral norm, conditional GAN with digit labels (forces all 10 classes), monitor diversity with FID/class histogram.

Case: MRI GAN – collapse, unstable loss, artifacts, missing detail (Q30)

Extra fixes for fine detail: progressive growing / StyleGAN2, PatchGAN discriminator, perceptual loss, spectral norm, higher resolution, radiologist validation, privacy check (no memorised patient scans). Consider diffusion models as a modern alternative.

DCGAN (Radford et al., 2015) replaces the fully connected layers of a vanilla GAN with convolutional layers, which preserve spatial structure, giving much better and more stable image generation.

Architecture

Generator:

z (100) → Project & reshape 4×4×1024 → ConvT 8×8×512 → ConvT 16×16×256 → ConvT 32×32×128 → ConvT 64×64×3 + Tanh

Discriminator:

Image 64×64×3 → Conv s2 32×32×128 → Conv s2 16×16×256 → Conv s2 8×8×512 → Conv s2 4×4×1024 → Flatten + Sigmoid

Architecture guidelines (memorise these 5)

1. Replace pooling with strided convolutions (D) and transposed (fractionally-strided) convolutions (G) — network learns its own up/down-sampling.
2. Use Batch Normalisation in both G and D (except G output & D input layers).
3. Remove fully connected hidden layers.
4. G: ReLU in all layers, Tanh at output.
5. D: LeakyReLU (0.2) in all layers, sigmoid at output. Adam lr = 0.0002, β₁ = 0.5.

Vanilla GAN vs DCGAN

Transposed convolution: opposite of convolution — increases spatial size (upsampling) using learnable filters. Output size = (in − 1)·stride − 2·pad + kernel.

A Conditional GAN (Mirza & Osindero, 2014) gives extra information y (class label, text, image) to both the Generator and the Discriminator, so we can control what is generated (e.g., "generate digit 7").

Conditional objective

min_G max_D V = E_(x,y)[ log D(x | y) ] + E_(z,y)[ log(1 − D(G(z | y) | y)) ]

D now checks two things: (1) is it realistic? (2) does it match the label? So G must produce a realistic image of the requested class.

Ways to inject the condition (Q35)

• Concatenation: one-hot y concatenated with z (G) and with flattened image (D).
• Embedding: label → learned embedding vector → concatenate / multiply with z.
• Extra channels: for conv D, label expanded to H×W×C maps and stacked with image channels.
• Projection discriminator: D output = ψ(φ(x)) + embed(y)^T φ(x).
• Conditional BatchNorm / AdaIN: label controls normalisation scale & shift.
• Auxiliary classifier (AC-GAN): D also predicts the class.
• Image as condition (Pix2Pix): input image concatenated with output image for D.

MNIST example (Q33)

G input = 100-dim z + 10-dim one-hot (or embedding of 0–9) → 110 → dense layers → 784 Tanh. D input = 784 image + 10-dim label → dense → sigmoid. At test time: fix y = 3, change z → many different handwriting styles of "3".

GAN vs cGAN (Q34)

Applications: class-specific augmentation (rare diseases), text-to-image, image-to-image (Pix2Pix), face aging with age label, product images with chosen colour/style. Limitations: needs labels, can ignore the label if poorly designed, only controls seen conditions, still has GAN instability.

Image-to-image translation: Pix2Pix (Q44, Q45)

A cGAN where the condition is an input image (e.g. grayscale photo) and the output is the translated image (colour photo).

Grayscale image (L channel) → U-Net Generator (encoder–decoder + skip connections) → Coloured image → PatchGAN D sees (gray, colour) pair → Real/fake for each 70×70 patch

L = L_cGAN(G, D) + λ · E[ ‖ y − G(x) ‖_1 ],   λ = 100

• Generator (U-Net): encoder downsamples to capture context; decoder upsamples; skip connections carry edges/details directly.
• Discriminator (PatchGAN): judges local patches → sharp textures.
• L1 loss: keeps output close to ground truth (correct colours/structure); GAN loss adds realism/sharpness.
• Training: paired data (convert colour photos to grayscale to create pairs).
• E-commerce: product sketch → photo, change background/day→night, style transfer. If no paired data → CycleGAN (two G's, two D's, cycle-consistency loss ‖F(G(x)) − x‖).

StyleGAN (NVIDIA, Karras et al. 2019) is a style-based generator: instead of feeding noise z directly to the generator, z is first transformed by a mapping network into an intermediate latent w, and w controls the style at every resolution layer through AdaIN. This gives very high quality (1024×1024 faces) and fine control over coarse-to-fine attributes.

Key components (Q37–Q40)

• Mapping network: 8-layer MLP, z (512) → w (512). Z must follow a fixed Gaussian, which "entangles" features; W is learned freely, so features become disentangled (one direction ≈ one attribute).
• Learned constant input: synthesis network starts from a learned 4×4×512 tensor, not from z.
• AdaIN (style modulation): AdaIN(x_i, y) = y_(s,i) · (x_i − μ(x_i)) / σ(x_i) + y_(b,i). Normalise each feature map, then scale and shift using style from w. (StyleGAN2 replaces with weight modulation/demodulation to remove "droplet" artifacts.)
• Noise injection: random noise added per layer gives stochastic variation (hair placement, pores) without changing identity.
• Style mixing: use w₁ for coarse layers and w₂ for fine layers → pose of person A with colours/texture of person B. Also acts as regulariser.
• Truncation trick: w' = w̄ + ψ(w − w̄), ψ<1 → higher quality, less diversity.
• Progressive / multi-scale: built on ProGAN — trains from low to high resolution.

Traditional GAN vs StyleGAN (Q41, Q43)

Latent-space manipulation (face editing): find direction vector for "age" in W (e.g. via labelled samples / InterFaceGAN); w_new = w + α·d_age. Real photo → GAN inversion to find its w → edit → regenerate.

Fashion virtual studio design (Q42)

• Train StyleGAN2/3 on fashion model images (aligned, high-res).
• Coarse styles → pose, body shape, face structure. Middle → hairstyle, clothing shape, facial features. Fine → clothing texture/colour, background colour, lighting, micro detail. Noise → fabric wrinkles, hair strands.
• Style mixing: pose from model A + clothes texture from model B.
• Attribute control: discover semantic directions in W (hair length, smile), use conditional StyleGAN for clothing category, inversion to edit real catalogue photos.
• Risks: bias, identity misuse → watermark outputs.

Other modern variants: ProGAN (progressive growing), BigGAN (large-scale class-conditional, ImageNet), StyleGAN2 (weight demodulation, no droplets), StyleGAN3 (alias-free, smooth animation), CycleGAN, SRGAN, WGAN-GP, SAGAN (self-attention GAN).

A diffusion model learns to generate data by reversing a gradual noising process. Forward process: slowly add Gaussian noise to a real image over T steps until it becomes pure noise. Reverse process: a neural network (U-Net) learns to remove the noise step-by-step, so starting from random noise it can create a new image. (T ≈ 1000 steps)

Forward process (math)

q(x_t | x_t−1) = N( √(1−β_t) · x_t−1, β_t I )   β_t = noise schedule (e.g. 10^-4 → 0.02)
α_t = 1 − β_t,   ᾱ_t = α_1 · α_2 · … · α_t (cumulative product)
 
Closed form (jump directly to any t):   x_t = √ᾱ_t · x_0 + √(1 − ᾱ_t) · ε,   ε ~ N(0, I)

√ᾱ_t = signal coefficient, √(1−ᾱ_t) = noise coefficient. As t increases, ᾱ_t → 0, so signal fades and noise dominates. Note: ᾱ_t + (1−ᾱ_t) = 1 → variance stays controlled.

Training (noise prediction) & reverse sampling

Loss: L = E_(x_0, t, ε) [ ‖ ε − ε_θ( √ᾱ_t x_0 + √(1−ᾱ_t) ε , t ) ‖² ]
 
Sampling step: x_(t−1) = (1/√α_t) · ( x_t − (β_t/√(1−ᾱ_t)) · ε_θ(x_t, t) ) + σ_t z

1. Training: take real x_0 → pick random t → sample ε → make x_t with closed form → U-Net predicts ε → MSE loss between true and predicted noise → backprop.
2. Generation: start x_T ~ N(0,I) → for t = T…1: predict noise, subtract a bit, add small random noise → x_0 = new image.

Role of Gaussian noise (Q47): (1) destroys data gradually in a mathematically simple way (sum of Gaussians is Gaussian → closed form); (2) final state N(0,I) is easy to sample from; (3) predicting noise gives a simple, stable MSE objective; (4) random noise in reverse steps gives diversity.

✏️ Numerical type 1 (Q48 pattern)

Given: ᾱ_t = 0.64, x_0 = 3, ε = 1. (Use your exam's values — method is identical.)

(a) √ᾱ_t = √0.64 = 0.8;   √(1 − ᾱ_t) = √0.36 = 0.6
(b) x_t = 0.8 × 3 + 0.6 × 1 = 2.4 + 0.6 = 3.0
(c) Signal contribution = 0.8 × 3 = 2.4;   Noise contribution = 0.6 × 1 = 0.6
    (in variance terms: 64% signal, 36% noise)
(d) As t ↑, ᾱ_t ↓ (product of numbers < 1), so √ᾱ_t ↓ → signal shrinks, √(1−ᾱ_t) ↑ → noise grows. At t = T, ᾱ_T ≈ 0 ⇒ x_T ≈ ε (pure noise).

If exam gives β_t instead: α_t = 1 − β_t first, then ᾱ_t = product.

✏️ Numerical type 2 (Q49 pattern) — multi-step schedule

Given: β_1 = 0.1, β_2 = 0.2, β_3 = 0.3, x_0 = 2, ε = 0.5.

Analysis: signal fraction (ᾱ) falls 90% → 72% → 50.4%; noise fraction (1−ᾱ) rises 10% → 28% → 49.6%. By step 3 signal and noise are almost equal. With more steps the image becomes pure noise.

Stable Diffusion = Latent Diffusion + text conditioning (Q51)

Text prompt → Tokenizer + CLIP Text Encoder (77×768 embeddings) → Cross-attention into U-Net

Random latent 64×64×4 (noise) → U-Net denoises T steps (predicts ε, scheduler: DDIM/DPM) → Clean latent → VAE Decoder → 512×512 image

• VAE encoder/decoder: compresses 512×512×3 image into 64×64×4 latent (48× smaller) → diffusion runs in latent space → much faster and cheaper than pixel-space DDPM.
• Text encoder (CLIP): converts prompt into embeddings that capture meaning.
• U-Net: encoder–decoder with skip connections, residual blocks, time-step embedding, and cross-attention layers (Q = image latent features, K, V = text embeddings) → this is the conditioning mechanism.
• Classifier-free guidance (CFG): ε̂ = ε_uncond + s·(ε_cond − ε_uncond), s ≈ 7.5 → stronger prompt following.
• Training: image → VAE encode → add noise to latent → U-Net predicts noise given caption → MSE loss.
• Advantages: high quality, diverse, controllable by text, open-source, img2img, inpainting, ControlNet. Limitations: slow (many steps), struggles with text in images/hands/counting, bias, copyright & deepfake concerns, needs GPU.

Diffusion vs GAN (Q50)

Medical use — benefits: augment rare conditions, balance classes, share data without real patients. Challenges: very small data (fine-tune pretrained / use latent diffusion), clinical validity (hallucinated lesions), evaluation by radiologists, compute. Privacy risks: diffusion models can memorise and reproduce training images → membership-inference attacks; mitigate with deduplication, differential privacy (DP-SGD), nearest-neighbour checks, ethics approval.

VAE

import torch, torch.nn as nn, torch.nn.functional as F
from torchvision import datasets, transforms
loader = torch.utils.data.DataLoader(
    datasets.MNIST('.', train=True, download=True, transform=transforms.ToTensor()),
    batch_size=128, shuffle=True)
 
class VAE(nn.Module):
    def __init__(self, z_dim=20):
        super().__init__()
        self.fc1 = nn.Linear(784, 400)            # encoder
        self.fc_mu = nn.Linear(400, z_dim)        # mean
        self.fc_logvar = nn.Linear(400, z_dim)    # log variance
        self.fc3 = nn.Linear(z_dim, 400)          # decoder
        self.fc4 = nn.Linear(400, 784)
    def encode(self, x):
        h = F.relu(self.fc1(x)); return self.fc_mu(h), self.fc_logvar(h)
    def reparameterize(self, mu, logvar):
        std = torch.exp(0.5 * logvar)
        eps = torch.randn_like(std)               # randomness outside the network
        return mu + std * eps                     # z = mu + sigma * eps
    def decode(self, z):
        return torch.sigmoid(self.fc4(F.relu(self.fc3(z))))
    def forward(self, x):
        mu, logvar = self.encode(x.view(-1, 784))
        z = self.reparameterize(mu, logvar)
        return self.decode(z), mu, logvar
 
def vae_loss(x_hat, x, mu, logvar):
    recon = F.binary_cross_entropy(x_hat, x.view(-1, 784), reduction='sum')
    kl = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())
    return recon + kl
 
model = VAE(); opt = torch.optim.Adam(model.parameters(), lr=1e-3)
for epoch in range(10):
    for x, _ in loader:
        x_hat, mu, logvar = model(x)
        loss = vae_loss(x_hat, x, mu, logvar)
        opt.zero_grad(); loss.backward(); opt.step()
    print(epoch, loss.item() / len(x))
 
# Generate NEW digits: sample from prior N(0, I) and decode
with torch.no_grad():
    z = torch.randn(16, 20)
    samples = model.decode(z).view(-1, 1, 28, 28)   # show with matplotlib / save_image

GAN

tf = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))])  # → [-1, 1]
loader = torch.utils.data.DataLoader(datasets.MNIST('.', train=True, download=True, transform=tf),
                                     batch_size=128, shuffle=True)
z_dim = 100
G = nn.Sequential(nn.Linear(z_dim, 256), nn.LeakyReLU(0.2),
                  nn.Linear(256, 512), nn.LeakyReLU(0.2),
                  nn.Linear(512, 784), nn.Tanh())             # fake image in [-1,1]
D = nn.Sequential(nn.Linear(784, 512), nn.LeakyReLU(0.2),
                  nn.Linear(512, 256), nn.LeakyReLU(0.2),
                  nn.Linear(256, 1), nn.Sigmoid())            # P(real)
bce = nn.BCELoss()
optG = torch.optim.Adam(G.parameters(), lr=2e-4, betas=(0.5, 0.999))
optD = torch.optim.Adam(D.parameters(), lr=2e-4, betas=(0.5, 0.999))
 
for epoch in range(50):
    for real, _ in loader:
        b = real.size(0); real = real.view(b, -1)
        ones, zeros = torch.ones(b, 1), torch.zeros(b, 1)
        # ---- 1) train Discriminator: real -> 1, fake -> 0
        fake = G(torch.randn(b, z_dim))
        lossD = bce(D(real), ones) + bce(D(fake.detach()), zeros)
        optD.zero_grad(); lossD.backward(); optD.step()
        # ---- 2) train Generator: make D say 1 (non-saturating loss)
        lossG = bce(D(fake), ones)
        optG.zero_grad(); lossG.backward(); optG.step()
    print(f"epoch {epoch}  D={lossD.item():.3f}  G={lossG.item():.3f}")
    # save G(fixed_noise) every few epochs to show progress: noise -> blobs -> digits

Tip: Explain in exam: fake.detach() stops D's loss from updating G. Tanh output matches [−1,1] normalisation. Fixed noise vector is used to visualise improvement at epochs 1, 10, 50.

The Transformer ("Attention Is All You Need", Vaswani et al., 2017) is a neural network that processes a whole sequence in parallel using self-attention instead of recurrence. It is the backbone of LLMs (GPT, BERT, Claude, T5) and of text conditioning in image models.

Why Transformers? (motivation)

Full architecture (Q55) — the diagram above

Encoder layer (Q55, Q60, Q62)

1. Input embedding: each token → d_model-dim vector (512).
2. + Positional encoding (adds word order).
3. Multi-head self-attention: each word looks at all words and collects context.
4. Add & Norm: residual connection x + Sublayer(x), then LayerNorm.
5. Position-wise Feed-Forward: FFN(x) = max(0, xW₁ + b₁)W₂ + b₂, inner size 2048, applied to every token separately.
6. Add & Norm again. Repeat N = 6 times. Output = contextual embeddings.

• Residual connection: lets gradient flow directly through deep stacks, avoids vanishing gradient, keeps original info.
• Layer normalisation: normalises each token's features to mean 0, variance 1 → stable and faster training (unlike BatchNorm, independent of batch size).
• Encoder-only models: BERT, RoBERTa → understanding tasks (classification, NER, embeddings).

Contextual embeddings (Q60)

Sentences: (1) "I deposited money in the bank." (2) "We sat on the river bank." (3) "You can bank on me." The static word vector for "bank" is the same at input, but after self-attention it mixes with "money", "river", "on me" → three different output vectors (finance / land / rely). Cosine similarity between (1) and (2) "bank" vectors is low. This is why Transformers handle polysemy, unlike Word2Vec.

Text classification with an encoder (Q62)

"Movie was great" → Tokenizer: [CLS] movie was great [SEP] → Embeddings + PE → BERT encoder layers → [CLS] vector / mean pooling → Linear + Softmax → Positive 0.97

Training objective: cross-entropy loss between predicted class distribution and true label; fine-tune whole model (lr ≈ 2e-5).

Self-attention lets every token in a sentence look at every other token and decide how much attention to pay to each, then build a new representation as a weighted sum. Example: in "The animal didn't cross the street because it was tired", "it" attends strongly to "animal".

Query (Q) = what I am looking for. Key (K) = what I contain (label). Value (V) = actual information I give. (Like YouTube search: query = your search, keys = video titles, values = videos.)

Attention(Q, K, V) = softmax( Q Kᵀ / √d_k ) · V,   Q = XW^Q, K = XW^K, V = XW^V

X (embeddings) → Q, K, V = XW → Scores QKᵀ → ÷ √d_k → (mask) → Softmax (rows sum to 1) → × V → Output Z

Why divide by √d_k? Dot products grow with dimension; large values push softmax into extreme 0/1 regions with tiny gradients. Scaling keeps variance ≈ 1 → stable gradients.

✏️ Worked numerical (Q57) — 3 tokens, d = 2

X = [[1, 0], [0, 1], [1, 1]]   (tokens: "I", "love", "AI")
W^Q = [[1, 0], [1, 1]],   W^K = [[1, 1], [0, 1]],   W^V = [[1, 0], [0, 2]]

Step 1 – Q, K, V (row × matrix):

Step 2 – Scores S = QKᵀ (dot of each query with each key):

S = [[1, 0, 1], [2, 1, 3], [3, 1, 4]]
e.g. row 2: q=[1,1]·k₁[1,1]=2, ·k₂[0,1]=1, ·k₃[1,2]=3

Step 3 – Scale by √d_k = √2 = 1.414:

S/√2 = [[0.707, 0, 0.707], [1.414, 0.707, 2.121], [2.121, 0.707, 2.828]]

Step 4 – Softmax each row (e^s / Σe^s):

Row 1: e^0.707=2.028, e^0=1 → sum 5.056 → [0.401, 0.198, 0.401]
Row 2: subtract max 2.121 → e: [0.493, 0.243, 1] sum 1.736 → [0.284, 0.140, 0.576]
Row 3: subtract max 2.828 → e: [0.493, 0.120, 1] sum 1.613 → [0.306, 0.074, 0.620]
A = [[0.401, 0.198, 0.401], [0.284, 0.140, 0.576], [0.306, 0.074, 0.620]]   (each row sums to 1)

Step 5 – Output Z = A·V (weighted sum of value rows [1,0], [0,2], [1,2]):

z₁ = 0.401[1,0] + 0.198[0,2] + 0.401[1,2] = [0.802, 1.198]
z₂ = 0.284[1,0] + 0.140[0,2] + 0.576[1,2] = [0.860, 1.432]
z₃ = 0.306[1,0] + 0.074[0,2] + 0.620[1,2] = [0.926, 1.388]

Interpretation: "AI" pays 62% attention to itself and 31% to "I"; each output is now a context-aware vector. Visualise A as a 3×3 heatmap (rows = queries, columns = keys).

Tip: If the exam gives only X (no W), take W = I so Q = K = V = X, and follow the same 5 steps.

Instead of one attention, run h attention "heads" in parallel, each with its own W^Q, W^K, W^V, so each head can focus on a different type of relationship. Then concatenate and project.

head_i = Attention( X W_i^Q, X W_i^K, X W_i^V )
MultiHead(X) = Concat(head₁, …, head_h) · W^O
Original: d_model = 512, h = 8, d_k = d_v = 512/8 = 64

• Why multiple heads? One softmax averages everything into one view. Multiple heads capture different relations simultaneously (grammar, meaning, position, coreference) in different subspaces → richer representation.
• Same total compute as single full-size head (because each head is d/h).
• Heads run in parallel → efficient on GPUs. Some heads become redundant (can be pruned).

Self-attention is permutation-invariant — "dog bites man" and "man bites dog" would look identical. Positional encoding adds a unique vector for each position to the token embedding so the model knows word order.

PE(pos, 2i) = sin( pos / 10000^(2i / d_model) )
PE(pos, 2i+1) = cos( pos / 10000^(2i / d_model) )
 
Input to Transformer = Embedding(token) + PE(pos)   (element-wise addition, same dimension)

Example (d_model = 4, pos = 1): i = 0 → angle = 1/10000⁰ = 1 → sin 1 = 0.841, cos 1 = 0.540; i = 1 → angle = 1/10000^(2/4) = 1/100 = 0.01 → sin = 0.010, cos = 1.000. PE(1) = [0.841, 0.540, 0.010, 1.000]. For pos = 0: [0, 1, 0, 1].

Why sinusoids?

• Each position gets a unique pattern; values bounded in [−1, 1].
• Different dimensions have different wavelengths (2π to 10000·2π) → low dims change fast (fine position), high dims slow (coarse position) — like clock hands.
• Relative position: PE(pos + k) is a linear (rotation) function of PE(pos) → model can learn relative distances.
• No parameters; can extrapolate to longer sequences than seen in training.
• Alternatives: learned positional embeddings (BERT, GPT-2), RoPE (rotary, LLaMA), ALiBi.

Cross-attention is attention between two different sequences: Queries come from the decoder (what is being generated), Keys and Values come from the encoder output (the source). It lets the decoder "look back" at the input while generating each output token.

CrossAttn = softmax( Q_dec K_enc^T / √d_k ) V_enc

Translation example: English "I love India" → Hindi "मैं भारत से प्यार करता हूँ". When the decoder is generating "भारत", its query matches the key of "India" most strongly → it takes the value (meaning) of "India". This gives soft alignment between source and target words.

Decoder layer

1. Masked (causal) multi-head self-attention: token t can only see tokens 1…t. Implemented by adding −∞ to scores above the diagonal before softmax → those weights become 0. Prevents "cheating" by looking at future words.
2. Add & Norm.
3. Cross-attention to encoder output (only in encoder–decoder models like T5; GPT-style decoder-only models skip it).
4. Add & Norm → Feed-forward → Add & Norm.
5. Output projection: Linear layer d_model → vocabulary size, then softmax → probability of every possible next token.

Causal mask M (4 tokens): [[0, −∞, −∞, −∞], [0, 0, −∞, −∞], [0, 0, 0, −∞], [0, 0, 0, 0]]
 
Masked attention = softmax( (QK^T / √d_k) + M ) V

Autoregressive generation

Prompt "The sky is" → Tokenize → IDs → Decoder stack → Softmax over vocab → Pick "blue" → Append & repeat → Stop at EOS / max length

Training: teacher forcing — feed correct previous tokens, predict next token, cross-entropy loss: L = −Σ_t log P(x_t | x_<t). All positions trained in parallel thanks to the mask.

Decoding strategies

An embedding is a dense numeric vector (e.g. 384–3072 numbers) that represents the meaning of a token, sentence or document. Similar meanings → nearby vectors.

• One-hot (sparse, no meaning) → static embeddings (Word2Vec, GloVe: one vector per word) → contextual embeddings (BERT/GPT: vector depends on sentence).
• Famous property: vec(king) − vec(man) + vec(woman) ≈ vec(queen).
• In LLMs: token IDs → embedding matrix lookup (vocab × d_model) → transformer layers → final hidden states.
• Sentence embeddings (Sentence-BERT, OpenAI/Cohere embedding models) used for search, clustering, RAG.

Cosine similarity = (A · B) / (‖A‖ ‖B‖)   ∈ [−1, 1];   1 = same direction (same meaning)

Example: A = [1, 2], B = [2, 4] → cos = 10 / (√5 · √20) = 10/10 = 1. Other metrics: dot product, Euclidean distance. Stored and searched in vector databases (FAISS, Chroma, Pinecone, Milvus) using approximate nearest neighbour (HNSW).

Prompt engineering is designing the input text to an LLM so that it produces the desired output — without changing the model's weights (in-context learning).

Same task, three styles: sentiment classification

ZERO-SHOT:
Classify the sentiment of this review as Positive or Negative.
Review: "The battery dies in two hours."
Sentiment:
 
ONE-SHOT:
Review: "Amazing camera quality!"  Sentiment: Positive
Review: "The battery dies in two hours."  Sentiment:
 
FEW-SHOT:
Review: "Amazing camera quality!"  Sentiment: Positive
Review: "Screen cracked on day one."  Sentiment: Negative
Review: "Fast delivery and great price."  Sentiment: Positive
Review: "The battery dies in two hours."  Sentiment:

Chain-of-Thought (CoT)

Q: A shop has 23 apples, uses 20 and buys 6 more. How many now?
A: Let's think step by step. 23 − 20 = 3. 3 + 6 = 9. Answer: 9.

Asking the model to show intermediate reasoning ("Let's think step by step") improves maths, logic and multi-step questions. Variants: few-shot CoT, self-consistency (sample many chains, majority vote), Tree-of-Thought.

Comparison of prompting types

Good prompt recipe: Role + Task + Context + Constraints + Output format + Examples.

RAG combines a retriever (search over your documents using embeddings) with a generator (LLM). Relevant chunks are retrieved and put into the prompt, so the LLM answers using up-to-date, private, verifiable information and cites sources — reducing hallucination without retraining.

The diagram above shows the online query-time path. Offline, a matching indexing pipeline runs first: Documents → Load & clean → Chunking → Embedding model → Vector DB, so that the query-time similarity search above has something to search against.

Step-by-step workflow

1. Ingestion: load PDFs, web pages, notices; clean text; attach metadata (title, date, page, department).
2. Chunking: split into pieces (e.g. 300–800 tokens) with overlap (10–20%) so meaning isn't cut. Strategies: fixed size, recursive by paragraphs/headings, semantic chunking.
3. Embedding generation: each chunk → vector using embedding model (e.g. all-MiniLM, text-embedding-3).
4. Indexing: store vectors + text + metadata in a vector database with ANN index (HNSW).
5. Retrieval: embed user query with the same model → cosine similarity → top-k chunks. Improve with hybrid search (BM25 keyword + vector) and a re-ranker (cross-encoder). Metadata filters (e.g. "2026 notices only").
6. Prompt augmentation: "Answer ONLY using the context below. If not present, say 'not found'. Cite sources. Context: [chunk1 (source)] … Question: …"
7. Generation: LLM writes the answer.
8. Source grounding: show citations (document name, page/section) so users can verify.

University knowledge-portal design

• Sources: syllabus PDFs, academic regulations, exam guidelines, notices (with dates).
• Chunking: by section headings (e.g. "Attendance rule 4.2"), 500 tokens, 50 overlap; metadata = doc type, program, semester, date, URL.
• Vector store: Chroma/FAISS/pgvector; re-index automatically when new notices are uploaded; latest notice overrides older (date filter).
• Retrieval: hybrid + re-rank, filter by student's program.
• Prompt: role "university assistant", answer only from context, cite, say "contact office" if unsure.
• Output: "Minimum attendance is 75% [Academic Regulations 2025, §4.2]".
• Evaluation: retrieval recall@k, answer faithfulness, relevance (RAGAS), human feedback. Access control for internal docs.

Applications & benefits

Customer support bots, enterprise document Q&A, legal/medical search, university helpdesk, code-base assistants. Benefits: fresh knowledge without retraining, fewer hallucinations, citations, private data stays in your DB, cheaper than fine-tuning. Challenges: bad chunking/retrieval → bad answers, context-window limits, latency, outdated index, prompt injection from documents.

# Minimal RAG (LangChain-style pseudocode)
chunks   = RecursiveTextSplitter(chunk_size=500, overlap=50).split(load_pdfs("docs/"))
db       = Chroma.from_documents(chunks, embedding=HuggingFaceEmbeddings("all-MiniLM-L6-v2"))
docs     = db.similarity_search(question, k=4)
context  = "\n\n".join(f"[{d.metadata['source']}] {d.page_content}" for d in docs)
prompt   = f"Answer only from the context and cite sources.\nContext:\n{context}\nQuestion: {question}"
answer   = llm.invoke(prompt)

How LLMs are trained

Pre-training: next-token prediction on trillions of tokens → Supervised fine-tuning (instructions) → RLHF / preference tuning → Deployment

• Training tricks: Adam/AdamW, learning-rate warm-up then decay, dropout, label smoothing, gradient clipping, mixed precision (FP16/BF16).
• Scaling laws: loss falls predictably as parameters, data and compute grow (power law). Chinchilla: ~20 training tokens per parameter for compute-optimal.

Challenges → solutions

Text generation with tokenization, inference, decoding

from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2")
 
prompt = "Artificial intelligence will change education by"
ids = tok(prompt, return_tensors="pt")           # 1) tokenization -> input_ids
print(ids["input_ids"])                          #    e.g. tensor([[8001, 9542, 4430, ...]])
 
greedy = model.generate(**ids, max_new_tokens=40, do_sample=False)                 # greedy
temp   = model.generate(**ids, max_new_tokens=40, do_sample=True, temperature=0.8)  # temperature
topk   = model.generate(**ids, max_new_tokens=40, do_sample=True, top_k=50)         # top-k
topp   = model.generate(**ids, max_new_tokens=40, do_sample=True, top_p=0.9)        # nucleus
for name, out in [("greedy", greedy), ("temp", temp), ("top-k", topk), ("top-p", topp)]:
    print(name, ":", tok.decode(out[0], skip_special_tokens=True))   # 3) detokenize

Observation to write: greedy = coherent but repetitive; temperature 1.2 = creative but can drift; top-k/top-p = best balance of diversity and coherence.

Summarization

from transformers import pipeline
summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
text = """Generative AI models learn the distribution of data ... (input paragraph)"""
print(summarizer(text, max_length=60, min_length=20, do_sample=False)[0]["summary_text"])

How it finds important info: BART/T5 is an encoder–decoder. The encoder's self-attention builds context for every sentence; while generating, cross-attention gives high weight to the most informative parts (named entities, main claims); it was fine-tuned on article→summary pairs (CNN/DailyMail) so it learned what humans keep. This is abstractive (writes new sentences) vs extractive (copies sentences).

Classification

clf = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")
print(clf("The lecture on GANs was really clear!"))   # [{'label': 'POSITIVE', 'score': 0.99}]

Each plan tells you exactly what to write. Expand using the sections above.

Q1. Explain GAN working principle / architecture with diagram, minimax objective and training algorithm. (10–16 marks)

1. Definition + counterfeiter-police analogy.
2. Labelled diagram: z → G → fake; real x; both → D → P(real); feedback arrows (§4).
3. Value function min_G max_D V; explain each term; D* = p_(data)/(p_(data)+p_g); equilibrium D = ½, JSD.
4. Alternating training algorithm (5 steps); forward/backward passes.
5. G/D design table (Tanh, Sigmoid, LeakyReLU).
6. Role of latent z.
7. Applications + 2 challenges.

Q2. VAE: architecture, reparameterization trick, derive ELBO, role of KL. (16 marks)

1. Definition; AE vs VAE quick table.
2. Diagram x → encoder → μ, logσ² → z = μ + σε → decoder → x̂.
3. Encoder, latent, decoder roles.
4. Reparameterization: why sampling blocks gradients; formula; ∂z/∂μ = 1.
5. ELBO derivation via Jensen (§3) + log p(x) = ELBO + KL(q‖p(z|x)).
6. Loss = Recon + KL; closed-form KL; why both needed.
7. Generation from N(0,I); 3 limitations (blurry, posterior collapse, loose bound).

Q3. Generative vs Discriminative models; Bayes; Naïve Bayes vs Logistic Regression. (7–16 marks)

1. Definitions; joint vs conditional with spam example.
2. Bayes derivation P(y|x) = P(x|y)P(y)/P(x).
3. Diagram (class blobs vs boundary).
4. 10-row comparison table.
5. NB formula vs LR sigmoid formula, training, decision, pros/cons.
6. Conclusion: generative for small data / generation / OOD; discriminative for accuracy.

Q4. GAN loss functions and training challenges (mode collapse, vanishing gradients, instability) + LSGAN / WGAN. (10–16 marks)

1. L_D, saturating L_G, non-saturating L_G with formulas.
2. Explain gradient issue: log(1−D) flat near 0 vs −log D steep near 0.
3. Three challenges with cause + symptom.
4. Fix table; LSGAN & WGAN-GP formulas.
5. Comparison table standard / LSGAN / WGAN; conclude WGAN-GP for medical images.

Q5. Diffusion models: forward & reverse process + numerical. (10 marks)

1. Definition; chain diagram x₀ → x_T and back.
2. q(x_t|x_(t−1)), α, ᾱ, closed form x_t.
3. Loss ‖ε − ε_θ‖²; training steps; sampling steps.
4. Numerical: compute α → ᾱ (product) → √ᾱ, √(1−ᾱ) → x_t → contributions → trend (§9 tables).
5. Role of Gaussian noise; diffusion vs GAN table.

Q6. Self-attention with numerical example (Q, K, V, softmax). (10 marks)

1. Concept + Q/K/V meaning + "it → animal" example.
2. Formula and flow diagram.
3. Five computational steps exactly as §12 (Q, K, V table → QKᵀ → ÷√d_k → softmax rows → AV).
4. Why √d_k; heatmap interpretation.

Q7. Transformer architecture (encoder, decoder, FFN, residual, LayerNorm) + positional encoding. (16 marks)

1. Motivation table (RNN problems).
2. Big encoder–decoder diagram (§11).
3. Encoder layer steps; decoder: masked self-attn, cross-attn, FFN, linear+softmax.
4. Formulas: attention, MHA, FFN, PE sin/cos.
5. Residual & LayerNorm purpose.
6. Variants: BERT (encoder), GPT (decoder), T5 (both).

Q8. Conditional GAN (+ MNIST digit control / image colourisation). (10 marks)

1. Definition; cGAN diagram with y into both G and D.
2. Objective with D(x|y), G(z|y).
3. Injection methods (concat, embedding, channels, projection).
4. MNIST example; GAN vs cGAN table.
5. Pix2Pix: U-Net + PatchGAN + L1 (for colourisation question).
6. Applications + limitations.

Q9. StyleGAN: mapping network, W space, AdaIN, coarse/middle/fine control, style mixing. (10–16 marks)

1. Key idea: style-based generator.
2. Diagram (§8): z → 8-layer MLP → w → A → AdaIN at each resolution; learned constant; noise.
3. AdaIN formula.
4. Coarse / middle / fine table with attributes.
5. Style mixing, truncation, stochastic variation.
6. Traditional GAN vs StyleGAN table; latent editing w + α·d.

Q10. RAG architecture / design for a university portal. (10–16 marks)

1. Definition + why (hallucination, fresh/private data).
2. Two-phase diagram (§19).
3. 8 steps: ingestion → chunking → embeddings → indexing → retrieval → prompt → generation → citations.
4. Specific design choices (chunk size, metadata, hybrid search, reranker, date filter).
5. Sample prompt template; evaluation; challenges.

Q11. Zero/one/few-shot and Chain-of-Thought prompting with examples. (7 marks)

1. Definition of prompt engineering & in-context learning.
2. Same-task examples (sentiment) for 0/1/few-shot.
3. CoT example (apples).
4. When-to-use table; comparison with role & instruction prompting.

Q12. Intro to GenAI: mechanism, types with applications, use cases & challenges, medical augmentation. (7–10 marks)

1. Definition (learns p(x), samples new data) + flow diagram.
2. Characteristics list.
3. Types table (VAE, GAN, Diffusion, Transformer, Flow) with one application each.
4. Domain use-case table; 2 challenges (hallucination, bias/privacy/deepfakes).
5. Medical augmentation justification.

Tip: Also prepare quickly: Multi-head attention (§13), cross-attention translation example (§15), DCGAN guidelines (§6), decoding strategies table (§16), Stable Diffusion pipeline (§9), code snippets (§10, §21) for any "implement" question — write the code skeleton + explain each block.