Real-life analogy: The open-book vs closed-book exam

Extractive QA is like an open-book exam where you must find and quote the exact sentence from the textbook that answers the question. Generative QA is like explaining the answer in your own words. Open-domain QA is like a closed-book exam — you must recall (or retrieve) relevant knowledge first, then reason about it. LLMs like GPT-4 do a mix: they have knowledge memorized in weights, but RAG gives them an open book.

Extractive QA — span prediction with BERT

Extractive QA models predict two token positions in the context: the start and end of the answer span. BERT-based models fine-tuned on SQuAD achieve near-human F1 scores by leveraging bidirectional context.

from transformers import pipeline

# BERT fine-tuned on SQuAD 2.0
qa = pipeline("question-answering",
    model="deepset/roberta-base-squad2")

context = """
The transformer architecture was introduced in the paper "Attention Is All
You Need" by Vaswani et al. in 2017. It replaced recurrent neural networks
with a self-attention mechanism, enabling parallelization and better
modeling of long-range dependencies. The encoder processes the input
sequence while the decoder generates the output sequence.
"""

questions = [
    "Who introduced the transformer architecture?",
    "What did transformers replace?",
    "What year was the transformer introduced?",
    "What does the encoder do?",
]

for q in questions:
    result = qa(question=q, context=context)
    print(f"Q: {q}")
    print(f"A: {result['answer']} (score: {result['score']:.2%})")
    print()

# Output:
# Q: Who introduced the transformer architecture?
# A: Vaswani et al. (score: 89.23%)
# Q: What year was the transformer introduced?
# A: 2017 (score: 96.41%)

Open-Domain QA and RAG

Open-domain QA requires retrieving relevant passages before answering — the system does not have a given context. The retrieval-augmented generation (RAG) pipeline:

1. Query encoding: Convert the question to a dense vector using a bi-encoder (e.g., DPR — Dense Passage Retrieval).
2. Retrieval: Search a vector database (FAISS, Pinecone, Chroma) for the top-k most similar document chunks using approximate nearest-neighbor search.
3. Reading / Generation: Pass the retrieved chunks + question to a reader model (BERT for extractive, GPT/BART for generative) to produce the final answer.

SQuAD and SQuAD 2.0: SQuAD (Stanford QA Dataset) has 100k+ Q&A pairs from Wikipedia. SQuAD 2.0 added 50k unanswerable questions (the answer is not in the passage) — models must also learn to say "I don't know" instead of always extracting a span. This tests reading comprehension more rigorously. EM (Exact Match) and F1 over answer tokens are the standard metrics.

Why extractive QA still exists in the LLM era

If GPT-class models answer any question in free text, why would anyone still fine-tune BERT to point at a span? Because the two approaches fail differently — and for some jobs, the older one is strictly better:

Where extractive QA is still the right call in 2026: Regulated document review (legal, medical, insurance), where the answer must be traceable to a highlighted source span; high-volume, low-latency search snippets; and any pipeline where a hallucinated answer is worse than no answer. A common production hybrid: extractive QA to locate and highlight the evidence span, an LLM to phrase the final answer from it — grounding the generation in a verifiable location rather than trusting it to quote correctly.

Evaluating QA: EM, F1, and what they miss

Exact Match (EM) is binary — the predicted string must equal a gold answer after normalization (lowercase, strip punctuation and articles). F1 is softer: it treats prediction and gold as bags of tokens and computes token-level precision and recall, so partial answers earn partial credit. Predicting "Gustave Eiffel" when the gold is "Eiffel" gives EM = 0 but F1 = 0.67.

Why EM/F1 break down for generative QA: Both metrics assume a short, canonical answer string — which is exactly what LLMs do not produce. A model answering "The tower was completed in 1889." against gold "1889" scores EM = 0 and a poor F1, despite being perfectly correct. This is why modern RAG evaluation abandons EM/F1 in favor of component-wise metrics: retrieval quality (recall@k, MRR, nDCG — did the right passage even get fetched?), faithfulness / groundedness (is every claim supported by the retrieved text?), and answer relevance (does it address the question?) — typically scored by an LLM judge and validated against human ratings. Frameworks like RAGAS and TruLens standardize this split. Diagnose retrieval before blaming the generator: most RAG failures are retrieval failures wearing a generation costume.

Practice questions

1. What are the two output tokens that an extractive QA model predicts? (Answer: Start token index and end token index of the answer span within the context passage.)
2. Why does RAG reduce hallucination compared to closed-book LLM QA? (Answer: RAG grounds the answer in retrieved documents — the model is conditioned on actual retrieved text, not solely on memorized training weights that may be outdated or incorrect.)
3. What does EM (Exact Match) measure in QA evaluation? (Answer: The percentage of predictions that exactly match the ground truth answer string after normalization (lowercase, remove punctuation). Strict metric — partial credit is given by token-level F1.)
4. DPR (Dense Passage Retrieval) uses a bi-encoder. What are the two encoders? (Answer: A question encoder and a passage encoder. Both trained so that relevant question-passage pairs have high dot-product similarity in embedding space.)
5. What makes SQuAD 2.0 harder than SQuAD 1.1? (Answer: SQuAD 2.0 includes unanswerable questions. Models must detect when no answer exists in the context instead of always extracting a span — requires reasoning about absence of evidence.)