Full Transformer architecture: encoder, decoder, attention

What's actually being asked

Explain the full Transformer architecture — encoder, decoder, self-attention, feed-forward networks, residual connections, and layer normalization.

Answer

High-level picture

A Transformer is an encoder–decoder stack built almost entirely out of attention and simple feed-forward layers — no recurrence, no convolution. The encoder turns an input sequence into contextual representations; the decoder generates the output sequence, attending both to its own previously-generated tokens and to the encoder's representations. The diagram above shows the full flow: source embeddings into the encoder stack, the encoder's output feeding the decoder as K/V, target embeddings into the decoder stack, and a final linear+softmax producing next-token probabilities.

Encoder

A stack of N identical layers. Each layer has two sub-blocks:

1. Multi-head self-attention — every position in the input attends to every other position (including itself), building a representation of each token in context of the whole sequence.
2. Position-wise feed-forward network (FFN) — a small 2-layer MLP (Linear → ReLU → Linear) applied independently to each position.

Both sub-blocks are wrapped with a residual connection followed by layer normalization: LayerNorm(x + Sublayer(x)). The encoder's final output is a sequence of contextual vectors, used as K and V for the decoder's cross-attention.

Decoder

Also a stack of N identical layers, but each layer has three sub-blocks:

1. Masked multi-head self-attention — attends only over the decoder's own previously-generated tokens; future positions are masked out so the model can't "cheat" by looking ahead during training.
2. Cross-attention — Query comes from the decoder's own sequence; Key and Value come from the encoder's output. This is how the decoder pulls information from the source sequence.
3. Position-wise FFN — same structure as the encoder's.

Each sub-block again gets LayerNorm(x + Sublayer(x)). The decoder's final output goes through a linear layer + softmax to produce next-token probabilities.

Self-attention (the mechanism inside both self-attention and cross-attention)

Q = XW_Q, K = XW_K, V = XW_V

Attention(Q,K,V) = softmax(QK^T / √d_k) V

• X — input sequence embeddings (one row per token).
• W_Q, W_K, W_V — learned projection matrices.
• QK^T — raw similarity scores between every pair of positions.
• √d_k scaling — d_k is the key dimension; dividing by its square root keeps dot products from growing too large and pushing softmax into near-zero-gradient regions.
• softmax turns scores into attention weights (sum to 1 per row); multiplying by V gives a weighted blend of value vectors — the output for each query position.

In self-attention, Q, K, V all come from the same sequence. In cross-attention (decoder's second sub-block), Q comes from the decoder sequence while K, V come from the encoder output.

Multi-head version runs h of these in parallel with separate projections, then concatenates:

head_i = Attention(QW_Q^i, KW_K^i, VW_V^i) MultiHead(Q,K,V) = Concat(head_1,...,head_h) W_O

Feed-forward network

Applied identically (same weights) to every position independently: a position-wise 2-layer MLP with a ReLU in between. It's where most of the model's per-token "thinking" capacity lives — attention itself is just a weighted average, it doesn't transform content; the FFN does.

Residual connections

Every sub-block's output is added back to its own input (x + Sublayer(x)) before normalization. This keeps gradients flowing cleanly through many stacked layers, so a 6- or 12-layer stack is actually trainable rather than suffering vanishing gradients — same reason residuals matter in any deep network.

Layer normalization

Normalizes activations across the feature dimension (per token, not per batch like BatchNorm), stabilizing the scale of activations flowing into the next sub-block. Combined with the residual connection, the pattern at every sub-block is: LayerNorm(x + Sublayer(x)).

Positional encoding (brief mention)

Since there's no recurrence or convolution, position information is injected separately — added to token embeddings before the first encoder/decoder layer. Sinusoidal form: PE(pos,2i) = sin(pos/10000^(2i/d_model)), PE(pos,2i+1) = cos(pos/10000^(2i/d_model)).

Exam tip

If the question just says "the Transformer," cover both encoder and decoder, and explicitly name the decoder's three sub-blocks (masked self-attn, cross-attn, FFN) — forgetting cross-attention, or forgetting the mask on decoder self-attention, is the most common point examiners dock.