Real-life analogy: The email triage clerk

Imagine a company receives thousands of emails daily. A clerk reads each one and sorts it into folders: Sales Inquiry, Technical Support, Billing, Spam, Complaint. This is multi-class text classification. The clerk learns patterns from experience: emails containing 'refund' and 'angry' go to Complaints; emails with 'free money' and 'click here' go to Spam. Machine learning models learn exactly these patterns — automatically, from labeled examples.

Naive Bayes for text classification

Naive Bayes is the classic text classifier. It applies Bayes theorem with the naive conditional independence assumption: given the class, all words are independent. Despite this unrealistic assumption, it works well for spam filtering and topic classification.

P(c \mid d) \propto P(c) \prod_{i=1}^{n} P(w_i \mid c)

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

# Sample data (in practice: 10k+ examples)
texts  = [
    "Win free iPhone now click here",
    "Meeting at 3pm tomorrow in conference room",
    "Claim your prize you have been selected",
    "Can you review the quarterly report",
    "FREE CASH no credit check apply now",
    "Hi please find attached the project update",
]
labels = ["spam", "ham", "spam", "ham", "spam", "ham"]

X_train, X_test, y_train, y_test = train_test_split(
    texts, labels, test_size=0.3, random_state=42)

# Pipeline: TF-IDF vectorization + Naive Bayes
clf = Pipeline([
    ("tfidf", TfidfVectorizer(ngram_range=(1, 2))),  # unigrams + bigrams
    ("nb",    MultinomialNB(alpha=0.1)),               # Laplace smoothing
])
clf.fit(X_train, y_train)
print(classification_report(y_test, clf.predict(X_test)))

# Predict new email
print(clf.predict(["Congratulations! You won $1000 click to claim"]))  # ['spam']

Sentiment analysis — beyond binary

Binary sentiment: positive / negative. Fine-grained sentiment: 1-5 star rating prediction. Aspect-based sentiment analysis (ABSA): 'The food was great but the service was terrible' — positive on food aspect, negative on service aspect. ABSA requires identifying both the aspect term and its polarity.

from transformers import pipeline

# Zero-shot sentiment (no fine-tuning needed)
sentiment = pipeline("sentiment-analysis",
    model="distilbert-base-uncased-finetuned-sst-2-english")

texts = [
    "This product is absolutely amazing, I love it!",
    "Worst purchase I have ever made. Do not buy.",
    "It was okay, nothing special but does the job.",
]
for text in texts:
    result = sentiment(text)[0]
    print(f"{result['label']:<10} ({result['score']:.2%})  {text[:50]}")

# Output:
# POSITIVE   (99.98%)  This product is absolutely amazing...
# NEGATIVE   (99.91%)  Worst purchase I have ever made...
# NEGATIVE   (52.38%)  It was okay, nothing special...

# For ABSA: use aspect-specific models or prompt LLMs
from transformers import pipeline as llm_pipeline
# "Classify the sentiment toward [food] in: 'food great service terrible'"

Evaluation metrics for classification

\text{Precision} = \frac{TP}{TP+FP} \quad \text{Recall} = \frac{TP}{TP+FN} \quad F_1 = \frac{2 \cdot P \cdot R}{P + R}

Accuracy is misleading for imbalanced data: If 99% of emails are ham and 1% are spam, a classifier that always predicts ham gets 99% accuracy but 0% recall on spam. Always report precision, recall, and F1 per class. For medical diagnosis, recall (sensitivity) is critical — missing a disease (false negative) is worse than a false alarm.

Classification in the LLM era: prompt, fine-tune, or distill?

In 2026 the practical question is rarely "which classical algorithm?" — it is whether to call an LLM at all. Prompting a frontier model gives instant, label-free classification at ~90%+ quality, but at per-request cost and 100–1000× the latency of a small fine-tuned model. The standard playbook:

The rule of thumb: If the task is stable, the labels are fixed, and volume exceeds a few thousand predictions per day — a fine-tuned small model (often trained on LLM-generated labels) beats calling an LLM on cost, latency, AND consistency. If labels change weekly or volume is small, stay on the LLM API. Most production sentiment systems in 2026 are cascades: a distilled classifier in front, an LLM behind it for the ambiguous 5–10%.

Production concerns the tutorials skip

• Class imbalance: real spam/abuse/churn datasets are 1–5% positive. Fixes in order of preference: class weights (class_weight='balanced'), threshold tuning on precision-recall curves, then resampling (SMOTE last — it can fabricate unrealistic text-feature combinations).
• Threshold ≠ 0.5: the operating threshold is a business decision. Plot precision vs recall and pick the point matching the cost of each error type; a fraud model and a content-recommendation model should never share a threshold.
• Calibration: a model that says 0.9 should be right ~90% of the time. Neural classifiers are systematically overconfident — apply temperature scaling or isotonic regression before anyone treats scores as probabilities.
• Label noise: crowd-sourced sentiment labels disagree 10–20% of the time (annotator agreement caps your ceiling). Measure inter-annotator agreement (Cohen's kappa) before blaming the model.
• Drift: vocabulary shifts (new slang, product names, events) silently degrade text classifiers. Monitor prediction distribution and confidence over time; retrain on a schedule or on drift alarms — see the Distribution Shift entry.

Practice questions

1. Naive Bayes classifies "free money" as spam. The word "free" never appeared in ham training data. What problem occurs? (Answer: Zero probability — P("free"|ham) = 0, making the entire product 0. Solution: Laplace (add-1) smoothing adds 1 to all word counts.)
2. A spam filter has precision 0.95, recall 0.70. What does this mean? (Answer: Of predicted spam, 95% are actually spam (low false positives). But it misses 30% of actual spam (low recall). Better to tune toward high recall for spam.)
3. Why is BERT better than TF-IDF + SVM for sentiment on "I am not unhappy"? (Answer: TF-IDF misses double negation. BERT reads the full contextual sequence bidirectionally and understands "not unhappy" ≈ positive.)
4. What is aspect-based sentiment analysis? Give an example. (Answer: Identifying sentiment at the entity level. "The battery life is poor but the screen is gorgeous" → battery: NEGATIVE, screen: POSITIVE.)
5. Name two evaluation metrics besides accuracy for text classification. (Answer: F1 score (harmonic mean of precision and recall) and AUROC (area under the ROC curve — measures discrimination ability at all thresholds).)