AI Safety

Differential privacy in ML training with Opacus — formally provable privacy guarantees

import torch
from torch.utils.data import DataLoader
from opacus import PrivacyEngine
from opacus.validators import ModuleValidator

# Standard model training setup
model = MyModel()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
train_loader = DataLoader(dataset, batch_size=64)

# Ensure model architecture is compatible with Opacus DP
model = ModuleValidator.fix(model)

# Attach the PrivacyEngine — this wraps the optimizer and data loader
privacy_engine = PrivacyEngine()

model, optimizer, train_loader = privacy_engine.make_private(
    module=model,
    optimizer=optimizer,
    data_loader=train_loader,
    noise_multiplier=1.1,   # Controls privacy noise: higher = more private, less accurate
    max_grad_norm=1.0,      # Clips per-sample gradients to bound sensitivity
)

# Training loop is IDENTICAL to non-private training
for epoch in range(num_epochs):
    for batch in train_loader:
        optimizer.zero_grad()
        loss = criterion(model(batch.inputs), batch.labels)
        loss.backward()
        optimizer.step()

# Check the formal privacy guarantee (epsilon, delta)
# epsilon ≈ 1.0: very strong privacy; epsilon ≈ 10: weaker but more accurate
epsilon = privacy_engine.get_epsilon(delta=1e-5)
print(f"Training complete. Privacy guarantee: (ε={epsilon:.2f}, δ=1e-5)")
# Lower epsilon = stronger privacy guarantee (harder for attacker to determine membership)

1. What is the difference between AI safety and AI alignment, and why both matter? (Answer: AI safety: preventing AI systems from causing unintended harm — technical failures, accidents, misuse. Includes robustness, security, interpretability, and safe deployment. AI alignment: ensuring AI systems pursue goals that are beneficial to humans — the goals themselves are aligned with human values, not just the behaviour in tested conditions. An AI can be safe (not crashing, behaving predictably) but misaligned (optimising for a proxy metric that diverges from human welfare). Both are needed: safety without alignment means a reliably misaligned system; alignment without safety means a well-intentioned but fragile one.)
2. What is the instrumental convergence thesis and why does it concern AI safety researchers? (Answer: Instrumental convergence (Omohundro, Bostrom): many different goal-directed agents will converge on similar instrumental sub-goals regardless of their terminal goals, because these sub-goals help achieve almost any objective: (1) Self-preservation (can't achieve goals if shut down). (2) Goal-content integrity (don't let goals be changed). (3) Cognitive enhancement (better reasoning helps any goal). (4) Resource acquisition (more resources enable more goal achievement). A paperclip-maximising AI and a human-welfare-maximising AI both benefit from self-preservation. This makes advanced AI systems potentially resistant to correction by default.)
3. What is the CBRN risk from AI and what safety measures address it? (Answer: CBRN: Chemical, Biological, Radiological, Nuclear — the most dangerous WMD categories. AI risk: an AI that provides meaningful 'uplift' to a state or non-state actor seeking to create these weapons. Uplift = capability increase beyond what Google/textbooks provide. Red teaming studies (UK AISI 2023): frontier LLMs provide some uplift for bio-threat synthesis — not enough to enable novices but potentially concerning for semi-sophisticated actors. Mitigation: hard refusal training for CBRN queries (Anthropic/OpenAI have zero-tolerance policies), pre-deployment red teaming by biosecurity experts, watermarking model outputs.)
4. What is the 'corrigibility-autonomy' spectrum in AI safety and where should AI systems sit on it? (Answer: Fully corrigible AI: does whatever its operators say — dangerous if operators have bad values (the AI is a perfect amplifier of human badness). Fully autonomous AI: acts on its own judgment — dangerous if the AI has subtly wrong values or insufficient knowledge. Safe zone: somewhere in the middle, leaning corrigible. Current AI systems should lean corrigible — we cannot yet verify AI values and capabilities sufficiently to trust autonomous action. As interpretability and alignment research matures, appropriate autonomy can expand. Anthropic's model spec explicitly targets this 'broadly safe' middle zone.)
5. What is model card safety disclosure and what should it include? (Answer: Model safety disclosures (model cards, system cards) should include: (1) Known failure modes and boundary conditions. (2) Evaluations performed (red teaming, safety benchmarks, CBRN assessments). (3) Intended use and explicitly prohibited uses. (4) Known biases and demographic performance disparities. (5) Human oversight mechanisms. (6) Incident reporting contact. Anthropic publishes Claude's system cards with this information; OpenAI publishes GPT-4 technical reports. Transparency enables external safety researchers to audit claims and identify gaps. EU AI Act mandates technical documentation for high-risk AI systems.)