Skip to content
← Generative AI

Quick recap

93 stops, one path — each topic followed by the questions that test it. Everything you need is right on each stop — scroll to light up the trail.

  1. 01

    Introduction to Generative AI – Use Cases and Types

    Generative AI models learn the underlying data distribution so they can produce new, original samples — images, text, audio — rather than just classify or score existing ones.

  2. Q01

    Generative AI mechanism for medical image synthesis

    Generative models learn P(x) and sample from it; diffusion/VAE are more stable than GANs for augmenting small medical datasets.

  3. Q04

    Joint vs conditional probability with applications

    Joint P(x,y) is both events together; conditional P(y|x) narrows to cases where x is known — linked by P(x,y) = P(y|x)·P(x).

  4. 02

    Generative vs Discriminative Models

    A generative model learns the joint distribution P(x,y) and can generate new data; a discriminative model learns P(y|x) directly to separate classes, without modeling how the data itself was produced.

  5. 03

    Variational Autoencoders (VAE) – Concept, Architecture, and Loss

    A Variational Autoencoder is an autoencoder whose encoder outputs a probability distribution over the latent space instead of a single point, trained to both reconstruct its input and keep that distribution close to a standard normal, so new points sampled from it decode into realistic outputs.

  6. Q10

    Autoencoder vs variational autoencoder differences

    A VAE encodes to a distribution q(z|x) regularized toward N(0,I) via KL — missing in a plain AE — which is what makes sampling generate new data.

  7. Q11

    VAE components and ELBO loss derivation

    ELBO = E[log p(x|z)] − KL(q(z|x)‖p(z)); reconstruction keeps z informative while KL keeps the latent space sample-able — both terms are required.

  8. Q12

    KL-divergence term and latent space regularization in VAE

    KL term D_KL = −½Σ(1+log σ²−μ²−σ²) pulls every input's q(z|x) toward N(0,I), closing latent-space gaps so random z's decode to something sensible.

  9. Q13

    Reparameterization trick for VAE backpropagation

    Reparameterization trick: z = μ + σ⊙ε (ε~N(0,I)) moves randomness outside the gradient path, making the encoder end-to-end differentiable.

  10. Q14

    VAE architecture, ELBO derivation, and training losses

    ELBO derivation: log p(x) ≥ E[log p(x|z)] − KL(q(z|x)‖p(z)), via Jensen's inequality — maximizing it jointly trains encoder and decoder.

  11. Q15

    Critical analysis of VAE generation, ELBO, and limitations

    VAEs maximize a loose ELBO lower bound on log p(x); key limitations are blurry samples, a loose bound, posterior collapse, and no adversarial sharpening signal.

  12. Q16

    Designing a VAE pipeline for synthetic image generation

    A VAE's synthetic images are only proven useful for augmentation if they improve a downstream classifier's accuracy on a real, held-out test set.

  13. Q17

    Deriving the ELBO: posterior, prior, likelihood and KL

    ELBO = E_q(z|x)[log p(x|z)] − KL(q(z|x)‖p(z)); maximizing it raises a lower bound on log p(x) while tightening q(z|x) toward the true posterior.

  14. Q18

    VAE-based augmentation pipeline for a small image dataset

    VAE augmentation only counts if a classifier trained with filtered synthetic data beats a real-only baseline on an untouched real test set.

  15. Q22

    GAN architecture: forward/backward passes and the role of z

    G's backward pass must flow through D's frozen weights to update G's parameters; z is the GAN's only source of output diversity.

  16. Q43

    Synthetic faces: StyleGAN latent manipulation vs traditional GANs

    Editing w'=w+α·Δ_attr in StyleGAN's disentangled W isolates attributes like age or pose, unlike a single entangled z.

  17. Q69

    Variational Autoencoder on MNIST, step by step

    VAE's reparameterization trick z=μ+σ⊙ε keeps sampling differentiable; loss = BCE reconstruction + closed-form KL against N(0,I).

  18. Q70

    VAE image generation: sampling new MNIST digits

    After training, skip the encoder and sample z~N(0,I) directly to decode brand-new digits — valid because KL pushed q(z|x) toward that prior.

  19. 04

    Generative Adversarial Networks (GAN) – Concept and Applications

    A Generative Adversarial Network pits a Generator, which turns random noise into fake samples, against a Discriminator, which tries to tell real from fake — trained together until the Generator's fakes are indistinguishable from real data.

  20. Q19

    GAN minimax game: Generator vs Discriminator

    GAN objective: min_G max_D E[log D(x)] + E[log(1−D(G(z)))] — G never sees real data directly, only a gradient signal relayed through D.

  21. Q20

    Adversarial learning: how G and D interact in a GAN

    GAN: Generator (forger) and Discriminator (detective) train against each other, using each other's performance as the signal, not a labeled dataset.

  22. Q21

    The adversarial training loop and how G improves over time

    GAN training alternates D-then-G updates; G's gradient, relayed through the frozen, just-updated D, nudges it to chase D's constantly-moving decision boundary.

  23. Q23

    Vanilla GAN architecture and information flow

    Vanilla GAN: z→G→fake sample, {fake,real}→D→P(real), trained via min_G max_D E[log D(x)]+E[log(1−D(G(z)))] with no conditioning or stability add-ons.

  24. Q24

    Minimax optimization and alternating parameter updates in GANs

    D updates by gradient ascent, G by descent (or ascent on non-saturating log D(G(z))) — non-saturating avoids G's vanishing gradient when D(G(z))≈0.

  25. Q25

    GAN Generator and Discriminator architecture

    Each GAN iteration updates D's weights using D(x) and D(G(z)), then updates G's weights using a fresh pass through the just-updated, frozen D.

  26. Q26

    Standard GAN loss functions for G and D

    D's loss is BCE pushing D(x)→1 and D(G(z))→0; G minimizes log(1−D(G(z))) or (in practice) maximizes the non-saturating log D(G(z)), with equilibrium at D=0.5.

  27. Q27

    Comparing GAN loss variants and gradient quality

    Saturating G loss log(1−d) vanishes near d≈0; non-saturating −log d gives a strong gradient there instead — hence it's the practical default.

  28. Q28

    Three major GAN training challenges

    GAN's three core failures — mode collapse, vanishing gradients, instability — are each fixed at the loss level by WGAN, non-saturating loss, or LSGAN.

  29. Q29

    Diagnosing mode collapse in a digit GAN

    Mode collapse: G produces only a few digit styles while D grows falsely confident — fix with minibatch discrimination or a Wasserstein loss.

  30. Q30

    GAN objective comparison for synthetic MRI generation

    For synthetic MRI generation, WGAN-GP is the strongest fix for mode collapse, unstable loss curves, and artifacts; LSGAN is a lighter-weight fallback.

  31. Q31

    DCGAN architecture vs vanilla GAN

    DCGAN replaces vanilla GAN's FC layers with transposed-conv (G) and strided-conv (D) stacks plus BatchNorm, giving sharper, spatially coherent images.

  32. Q32

    Conditional GAN architecture and conditioning effect

    cGAN conditions both networks on y — G(z|y), D(x|y) — letting you choose what's generated, but conditioning adds control, not training stability.

  33. Q34

    Conventional GAN vs Conditional GAN comparison

    Conventional GANs learn only the marginal p(x) with no control; cGANs learn p(x|y), enabling controllable generation like image-to-image translation.

  34. Q41

    Traditional GANs vs modern variants: stability and control

    Non-saturating loss, progressive growth, and AdaIN style injection jointly fix GAN instability, quality, and control in StyleGAN.

  35. Q50

    Diffusion vs GAN for synthetic medical image generation

    Diffusion trains via stable noise-regression with better mode coverage than GAN's adversarial minimax; main privacy risk is memorizing small datasets.

  36. Q52

    GAN design for synthetic diseased-leaf images

    Conditional GAN G(z|y)/D(x|y) generates synthetic diseased-leaf images per class; the classifier must be validated only on real held-out leaves.

  37. Q53

    GAN design for synthetic network-flow traffic data

    Tabular GAN: categorical outputs need Gumbel-softmax (not argmax) for differentiability; evaluate with statistical/ML-efficacy metrics, not FID.

  38. Q54

    GAN architecture for synthetic satellite imagery

    Satellite-image GAN uses the standard value function; G trains with the non-saturating loss (maximize log D(G(z))) for stronger early gradients.

  39. Q71

    Basic GAN for generating handwritten digits

    Vanilla GAN on MNIST: G's Tanh output matches [-1,1]-normalized pixels; D trains on detached fakes, G trains with the non-saturating loss.

  40. 05

    GAN Architecture

    The Generator is an upsampling network that maps a latent noise vector to a full-sized sample, and the Discriminator is a downsampling classifier that outputs the probability its input is real.

  41. Q09

    Designing a multi-class image classifier: generative vs discriminative

    NN/SVM (discriminative) wins on raw image-classification accuracy; GMM/Naive Bayes (generative) uniquely enables out-of-distribution detection.

  42. Q36

    Complete cGAN architecture and training process

    cGAN training mirrors vanilla GAN's D-then-G alternation with y in every term; conditioning adds control, not immunity to mode collapse or instability.

  43. 06

    GAN Training Challenges and Loss Functions

    GANs are trained via a minimax game between Generator and Discriminator, which is why they're prone to mode collapse, vanishing gradients, and unstable loss curves — problems tackled with techniques like non-saturating loss and alternative objectives such as WGAN.

  44. 07

    Conditional GAN and Applications

    A cGAN feeds an extra conditioning signal — usually a class label — into both the Generator and Discriminator, so generation can be steered toward a specific category instead of being random.

  45. Q44

    GAN image-to-image translation for product photos

    Image-to-image translation = conditional GAN G(z|y)/D(x|y) with U-Net+PatchGAN+L1; CycleGAN's cycle-consistency loss handles unpaired data.

  46. 08

    StyleGAN & Modern GAN Variants

    StyleGAN replaces a single latent vector with a learned mapping network and injects "style" at every resolution of the Generator via AdaIN, so coarse features like pose and fine features like texture can be controlled independently.

  47. Q37

    StyleGAN's core idea vs traditional GANs

    StyleGAN injects w as "style" via AdaIN at every resolution, starting from a learned constant instead of feeding z once.

  48. Q38

    Mapping network, W space, and style modulation

    AdaIN(x,y)=y_s·norm(x)+y_b injects style per layer: coarse=pose/shape, middle=structure, fine=color/texture.

  49. Q39

    StyleGAN improvements: style control, quality, diversity

    StyleGAN beats traditional GANs via a disentangled W, per-resolution style control, and diversity dials (noise, style mixing, truncation).

  50. Q40

    From traditional GANs to StyleGAN: architecture trace

    Evolution: progressive growth → mapping network (Z→W) → per-layer style injection → per-pixel noise → free multi-scale control.

  51. Q42

    Virtual fashion studio with StyleGAN attribute control

    Map studio needs to style levels: pose/shape=coarse, hairstyle=middle, texture/color=fine, micro-detail=noise; style mixing crosses w_A/w_B.

  52. 09

    Diffusion Models (DDPM, Stable Diffusion) – Overview and Applications

    A diffusion model learns to reverse a fixed process that gradually adds Gaussian noise to data, training a network to predict and remove that noise one step at a time until pure noise becomes a realistic sample.

  53. Q46

    Diffusion models: forward noising and reverse denoising

    Diffusion's forward process (fixed) adds noise to x_0; the only learned part is the reverse U-Net predicting ε_θ(x_t,t) to denoise.

  54. Q47

    Gaussian noise's role in DDPM training and sampling

    DDPM predicts noise ε (not x_0) because Gaussian noise composes in closed form: x_t=√ᾱ_t·x_0+√(1−ᾱ_t)·ε.

  55. Q48

    DDPM forward diffusion: a worked 3-step example

    ᾱ_t is the cumulative product Πα_s, not 1−β_t — as t grows, the x_0 weight √ᾱ_t shrinks while the noise weight √(1−ᾱ_t) grows.

  56. Q49

    DDPM noise schedule: alpha-bar and signal-to-noise decay

    Signal-to-noise ratio √ᾱ_t/√(1−ᾱ_t) falls monotonically (3.0→1.0 in the toy example) since ᾱ_t is a product of terms below 1.

  57. Q51

    Stable Diffusion's text-to-image pipeline end to end

    Stable Diffusion denoises in a compressed VAE latent, conditioned on CLIP text via U-Net cross-attention, then VAE-decodes to the image.

  58. 10

    Practical Hands-on: GAN or VAE Implementation

    Implementing a GAN or VAE in PyTorch means defining the two (or encoder/decoder) networks, picking matching activation functions and loss terms, and training carefully — detaching the Generator's output before the Discriminator's step, and watching loss curves and sample grids rather than a single metric.

  59. 11

    Introduction to Transformers – Motivation for Generative AI

    Transformers replaced recurrent networks for sequence modelling because self-attention lets every token look directly at every other token in one parallel step, instead of passing information sequentially through hidden states.

  60. Q57

    Worked numeric example of single-head self-attention

    Attention(Q,K,V)=softmax(QK^T/√d_k)V — the final output is each token's weighted blend of V rows, weighted by its softmax row.

  61. 12

    Encoder Architecture

    A Transformer encoder block stacks multi-head self-attention and a feed-forward network, each wrapped in a residual connection and layer normalization, to build a contextual representation of every input token.

  62. Q55

    Full Transformer architecture: encoder, decoder, attention

    Every Transformer sub-block is LayerNorm(x+Sublayer(x)); the decoder adds masked self-attention and cross-attention atop self-attn+FFN.

  63. Q56

    Positional encoding: derivation and role in Transformers

    Sinusoidal PE(pos,2i)=sin(pos/10000^(2i/d_model)) is added to embeddings; angle-addition identities make PE(pos+k) linear in PE(pos).

  64. Q58

    Multi-head self-attention: mechanics and motivation

    Multi-head attention runs h heads each with its own W_Q/W_K/W_V in reduced dim d_k=d_model/h, then concatenates and projects via W_O.

  65. Q59

    Cross-attention: decoder using encoder context

    Cross-attention: Q comes from the decoder, K and V come from the encoder's output — the one fact examiners always check.

  66. Q61

    Transformer decoder architecture for text generation

    A causal mask sets QK^T entries where key-position>query-position to −∞, so GPT-style decoders only attend to past tokens and skip cross-attention.

  67. Q62

    Text classification using a Transformer encoder

    BERT-style classification: bidirectional self-attention (no causal mask) builds context, and the pooled [CLS] vector feeds a softmax classifier.

  68. Q63

    Text generation with a pretrained Transformer

    model.generate() loops one forward pass per new token, not one pass — do_sample+top_k/top_p sampling vs. deterministic greedy argmax.

  69. Q68

    Comparing greedy, temperature, and top-k/top-p decoding

    Greedy=deterministic argmax; temperature rescales logits; top-k/top-p truncate the tail before sampling for the best diversity/coherence balance.

  70. Q72

    Text summarization with a pretrained Transformer

    BART summarization uses beam search (num_beams=4); "importance" emerges from encoder self-attention plus decoder cross-attention, not explicit scoring.

  71. 13

    Prompt Engineering – Zero-shot, One-shot, Few-shot & Chain-of-Thought Prompting

    Prompt engineering shapes a model's output by how a request is phrased — from zero-shot (no examples) through few-shot (a handful of examples) to chain-of-thought (asking the model to reason step by step) — without changing any model weights.

  72. Q65

    Comparing instruction, role, and reasoning prompting styles

    Instruction, role, few-shot, and chain-of-thought prompting trade cost/simplicity for accuracy; CoT most helps multi-step reasoning tasks.

  73. 14

    Embeddings and Vector Representations for LLMs

    An embedding is a dense vector representation of a token or piece of text, placed so that semantically similar items sit close together in vector space, measured by metrics like cosine similarity.

  74. Q66

    RAG pipeline: from documents to grounded answers

    RAG embeds the query with the same model as the corpus, retrieves top-k chunks by cosine similarity, and grounds the LLM's answer in them.

  75. Q67

    RAG system design for a university portal

    University RAG tags chunks with doc_type/effective_date and boosts retrieval by recency, so amended regulations aren't shadowed by older ones.

  76. 15

    Retrieval-Augmented Generation (RAG) – Architecture and Applications

    Retrieval-Augmented Generation looks up relevant text chunks from an external knowledge store at query time and feeds them into the model's prompt, grounding its answer in real retrieved evidence instead of only its frozen training memory.

  77. 16

    Self-Attention – Concept, Step-by-Step Example, and Hands-on Visualization

    Self-attention computes a Query, Key, and Value vector for every token, then lets each token's output be a weighted blend of all tokens' Values — the weights set by how well its Query matches every Key.

  78. 17

    Multi-Head Attention

    Multi-head attention runs several smaller self-attention operations in parallel, each with its own learned projections, so different heads can specialise in different kinds of relationships between tokens.

  79. 18

    Positional Encoding

    Because self-attention has no built-in sense of order, positional encoding adds a unique sinusoidal (or learned) signal to each token's embedding based on its position in the sequence.

  80. 19

    Decoder Architecture

    A Transformer decoder block adds a causally-masked self-attention layer, so a token can only attend to earlier tokens, plus a cross-attention layer that lets it look at the encoder's output, before its own feed-forward sublayer.

  81. 20

    Cross-attention

    Cross-attention lets a decoder's Queries attend to Keys and Values produced by the encoder, which is how a translation model's output tokens stay grounded in the original input sentence.

  82. 21

    Transformer Training and Scaling Challenges

    Training large Transformers is expensive because self-attention cost grows quadratically with sequence length, which is why techniques like mixed-precision training, gradient checkpointing, and efficient attention variants matter at scale.

  83. Q02

    Four generative model types with real-world applications

    VAE→drug discovery, GAN→super-resolution, diffusion→text-to-image, autoregressive transformer→chat/code generation.

  84. Q03

    Generative AI use cases across six industries

    Across six industries, generative AI's two biggest adoption hurdles are hallucination/reliability and data privacy, bias, and IP/copyright risk.

  85. Q05

    Generative vs discriminative models comparison table

    Generative models learn P(x,y) and can generate new data; discriminative models learn P(y|x) directly and are usually more accurate at classification alone.

  86. Q06

    Mathematical foundations of generative vs discriminative learning

    Generative learning derives P(y|x) from P(x|y)P(y) via Bayes; discriminative learning fits P(y|x) directly, e.g. Naive Bayes vs. logistic regression on spam.

  87. Q07

    Bayes' theorem derivation and full generative-discriminative comparison

    Bayes' theorem: P(y|x) = P(x|y)P(y) / Σ_y' P(x|y')P(y') — derived by equating two expressions for the joint P(x,y).

  88. Q08

    Naive Bayes vs logistic regression on binary classification

    Naive Bayes models P(x,y) assuming conditionally independent features; Logistic Regression fits P(y|x) directly — both can reach the same linear boundary.

  89. Q33

    Class-conditioned digit generation with cGAN

    cGAN labels inject by concatenating y's embedding with z (into G) or x (into D); D rejects even a realistic image paired with the wrong label.

  90. Q35

    cGAN objective and conditioning-injection methods

    cGAN conditioning injects via input concatenation, conditional batch norm, feature-map concatenation, or a projection discriminator — key for text-to-image.

  91. Q45

    Colorizing photos with a GAN-based pix2pix approach

    Pix2pix colorization: U-Net generator + PatchGAN discriminator trained with adversarial + L1 loss on grayscale-to-color pairs.

  92. Q60

    Contextual embeddings: same word, different sentences

    Self-attention gives the same word different vectors in different sentences because it blends each token's output from that sentence's own K/V.

  93. Q64

    Zero-shot, one-shot, and few-shot prompting compared

    Zero/one/few-shot differ only in how many in-context examples sit in the prompt — none of the three updates any weights.