Text summarization with a pretrained Transformer

What's actually being asked

Use a pretrained encoder-decoder Transformer to summarize a paragraph with real code, show the input and the generated summary, and explain the mechanism by which the model decides what's "important" enough to keep.

Answer

Code

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
 
model_name = "facebook/bart-large-cnn"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
model.eval()
 
input_text = (
    "Generative AI refers to a class of machine learning models that learn the "
    "underlying distribution of training data and can then generate new, similar "
    "samples -- text, images, audio, or code. Unlike discriminative models, which "
    "learn a decision boundary between classes, generative models learn the full "
    "data distribution, which lets them synthesize novel content rather than just "
    "label existing content. This distinction is why the same underlying idea "
    "powers chatbots, image generators, and code assistants alike."
)
 
inputs = tokenizer(input_text, return_tensors="pt", max_length=1024, truncation=True)
 
summary_ids = model.generate(
    input_ids=inputs["input_ids"],
    attention_mask=inputs["attention_mask"],
    max_length=60,
    min_length=15,
    num_beams=4,
    length_penalty=2.0,
    early_stopping=True,
)
 
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
 
print("INPUT:\n", input_text)
print("\nSUMMARY:\n", summary)

Example output (illustrative — exact phrasing depends on the specific checkpoint/version, but this is the correct shape of what beam-search summarization with bart-large-cnn produces):

INPUT:
 Generative AI refers to a class of machine learning models that learn the
 underlying distribution of training data and can then generate new, similar
 samples -- text, images, audio, or code. ...
 
SUMMARY:
 Generative AI models learn the full data distribution of their training data,
 unlike discriminative models which only learn a decision boundary. This lets
 them generate new text, images, audio, or code.

What each part of the code is doing

• AutoModelForSeq2SeqLM loads an encoder-decoder Transformer (BART here — pretrained, then further fine-tuned on a large summarization dataset) — the right model class for summarization/translation, as opposed to AutoModelForCausalLM used for pure generation (Q63/Q68).
• The tokenizer truncates the input to the model's max input length (1024 tokens for BART) and returns input_ids + attention_mask.
• model.generate(...) with num_beams=4 uses beam search rather than greedy or sampling decoding — it keeps the 4 most promising partial summaries at each step instead of committing to a single best token, which tends to give more coherent, less repetitive summaries than greedy decoding. length_penalty and min_length/max_length control how long the summary is allowed to run.
• tokenizer.decode(..., skip_special_tokens=True) turns the output ids back into readable text.

How the model identifies "important" information

1. Encoder side: self-attention lets every token's representation be built from the entire input, both directions (no causal mask). Sentences carrying key entities, topic statements, or outcome claims naturally end up with representations that are strongly connected to (attended by) many other tokens in the document — the encoder isn't told explicitly what's important, but training pressure shapes its attention patterns to concentrate on content that recurs or anchors the rest of the passage.
2. Decoder side (cross-attention): at each step of generating the summary, the decoder's cross-attention — same formula as self-attention, but Q from the decoder, K/V from the encoder's output — effectively "queries" the encoder's representations to decide which part of the source is relevant for the next summary word. This is the actual mechanism by which salient spans of the input get pulled into the output.
3. Training objective: the model was trained (via next-token cross-entropy against human-written reference summaries, over a large summarization dataset) to predict exactly the kind of content humans chose to keep — topic sentences, named entities, outcome/conclusion statements — while learning to drop supporting detail, examples, and redundant phrasing. Over many such examples, the model generalizes a notion of "what summaries keep" purely from statistical exposure to that pattern, which is what then lets cross-attention correctly prioritize similar content at inference time on new, unseen paragraphs.

Exam tip

If asked to name the mechanism in one phrase: it's the combination of bidirectional self-attention in the encoder (builds globally-informed representations of every input token) and cross-attention in the decoder (lets each generated word selectively pull from the most relevant encoder representations) — not some separate "importance scoring" step bolted onto the architecture.