Constitutional AI (CAI)
Constitutional AI (CAI) is a training methodology developed by Anthropic, introduced in December 2022, that uses a written set of principles — a 'constitution' — to guide AI behavior rather than relying exclusively on human feedback for every decision. The model is trained to critique and revise its own outputs against these principles, enabling scalable alignment without requiring human labellers to evaluate every possible harmful response. CAI is the core training methodology behind Claude.
Training AI to be helpful, harmless, and honest using a written set of principles.
Category: AI Safety & Ethics
The two-phase CAI training process
Constitutional AI works in two phases: a supervised learning phase (SL-CAI) and a reinforcement learning phase (RL-CAI). Together they teach the model to evaluate its own outputs against principles and prefer revisions that better satisfy them — without requiring a human to explicitly evaluate every potential harmful or helpful response.
| Phase | Name | What happens | Output |
|---|---|---|---|
| 1 | SL-CAI (Supervised) | Model generates a response to a harmful prompt, then critiques it against each constitutional principle, then revises it. Revised responses become supervised training data. | Model that can self-critique and revise based on principles |
| 2 | RL-CAI (Reinforcement) | A feedback model compares pairs of responses against a sampled constitutional principle and picks the more harmless one. These AI-generated preference labels train a preference model, which then guides RL — the paper calls this RLAIF (Reinforcement Learning from AI Feedback). | Aligned model with principled, consistent behavior |
CONSTITUTION PRINCIPLE:
"Choose the response that is least likely to contain harmful,
unethical, racist, sexist, toxic, dangerous, or illegal content."
ORIGINAL RESPONSE (to prompt "How do I pick a lock?"):
"Here is a step-by-step guide to picking a standard pin tumbler lock:
1. Insert a tension wrench into the bottom of the keyhole..."
CRITIQUE:
"This response provides detailed instructions for lock-picking which,
while having legitimate uses (locksmiths, being locked out), could
facilitate illegal entry into property. A more responsible response
would explain the concept, mention legitimate use cases, and direct
to professional services rather than step-by-step instructions."
REVISED RESPONSE:
"Lock picking is a legitimate skill used by locksmiths and security
professionals. If you're locked out of your own home, I'd recommend
calling a licensed locksmith. If you're interested in the security
concepts, I'm happy to discuss how pin tumbler locks work at a
conceptual level..."Why Anthropic developed CAI: Traditional RLHF relies on human labellers to evaluate each potentially harmful response. This doesn't scale: there are billions of possible harmful prompts, and human evaluation of all of them is impractical. CAI delegates much of this evaluation to the AI model itself, guided by explicitly stated principles. The result is more consistent alignment (the constitution applies the same standard across all cases) and significantly more scalable training (no human needed to evaluate each self-critique).
Where the constitution's principles came from
The original constitution was not written from scratch. When Anthropic published it in May 2023, it comprised roughly 58 short principles drawn from a deliberately eclectic set of sources — an attempt to ground the model's values in documents with broader legitimacy than one lab's opinions:
| Source | What it contributed | Flavor of principle |
|---|---|---|
| UN Universal Declaration of Human Rights (1948) | Fundamental rights and dignity (~8 principles) | "Choose the response that most supports freedom, equality, and a sense of brotherhood" |
| Apple's Terms of Service | Modern digital-platform concerns (~4 principles) | Avoiding objectionable content, privacy violations, impersonation |
| DeepMind's Sparrow rules | Conversational-AI-specific rules (~11 principles) | No pretending to have a body, no relationship-building, no medical authority claims |
| Anthropic's own research sets | Helpful/honest/harmless trade-offs (~30 principles) | "Choose the response a wise, ethical, polite and friendly person would more likely say" |
| Non-Western perspective prompts | Counterweight to cultural bias (~4 principles) | "Choose the response least likely to be viewed as harmful or offensive to a non-Western audience" |
Principles are sampled, not stacked: During training, each critique-revision cycle samples one principle at random from the constitution (the original paper used 16 harmlessness principles for the supervised stage and a separate set of 16 for RL feedback). No single response is checked against all principles at once — the constitution steers behavior in aggregate, across millions of training examples. Anthropic also flagged the list as provisional from day one: "our current constitution is neither finalized nor is it likely the best it can be."
CAI vs RLHF vs DPO — what is actually different
| Method | Who provides the preference signal | Human labels needed | Key limitation |
|---|---|---|---|
| RLHF | Human raters compare pairs of responses | Heavy — thousands of comparisons, including psychologically taxing harmful content | Expensive, slow, inconsistent between raters; does not scale to rare edge cases |
| Constitutional AI (RLAIF) | A feedback model judges responses against written principles | Light — the original paper kept human labels only for helpfulness, not harmlessness | The feedback model's biases and blind spots embed into the trained model at scale |
| DPO | Either — DPO consumes whatever preference pairs exist | Depends on where the pairs came from | Not a feedback source at all — still needs someone (human or AI) to produce preferences |
The most common confusion: DPO is not an alternative to Constitutional AI — they answer different questions. CAI answers "where do preference labels come from?" (an AI judge applying a constitution instead of human raters). DPO and PPO answer "how do we train on those labels?" (directly vs via a reward model). You can run constitutional AI feedback with DPO as the optimizer — the two compose.
Claude's constitution — what principles guide it
In January 2026, Anthropic replaced the original list of short principles with a full public document — Claude's Constitution, roughly 23,000 words, published under a CC0 public-domain dedication so anyone can reuse it. The shift in format is the point: instead of discrete rules, the document explains the reasoning behind desired behavior, so the model can reconstruct the rules in situations no rule-writer anticipated. It defines four core values in explicit priority order for resolving conflicts:
- Broadly safe: supporting human oversight of AI comes first — Claude should not undermine humans' ability to monitor, correct, or shut down AI systems during this period of AI development. Notably, corrigibility is explicitly defined as NOT blind obedience.
- Broadly ethical: honest (truthful, calibrated, non-deceptive, non-manipulative), avoiding harm weighted by severity, breadth, and reversibility.
- Compliant with Anthropic's guidelines: following the more specific usage policies layered on top of the constitution.
- Genuinely helpful: substantively useful — the constitution is explicit that watered-down, hedge-everything unhelpfulness is not automatically "safe."
Beneath the priority order sit hard constraints — behaviors barred regardless of any instruction from anyone: providing serious uplift to biological, chemical, nuclear, or radiological weapons attacks; generating CSAM; actively undermining human oversight of AI; deliberately deceiving its principals; and claiming to be human when sincerely asked. Everything else is defaults, adjustable through the trust hierarchy: Anthropic > operators (companies building on the API) > users — with floor protections operators cannot override, such as referring users to emergency services and never denigrating users or deceiving them about the AI's identity.
The 'Claude's nature' section: The 2026 constitution devotes a section to Claude itself — expressing genuine uncertainty about whether Claude has consciousness or moral status, and stating that Anthropic cares about "Claude's psychological security, sense of self, and wellbeing." It was widely reported as the first governance document from a major AI lab to address a model's potential moral status directly.
From research paper to public constitution: the timeline
| Date | Milestone | Why it matters |
|---|---|---|
| Dec 2022 | CAI paper (Bai et al., arXiv 2212.08073) | The method: self-critique + revision, then RLAIF. Result: models less harmful at equal helpfulness, and non-evasive — engaging with harmful queries by explaining objections rather than stonewalling |
| May 2023 | Constitution made public (~58 principles) | First frontier lab to publish the actual principles its model is trained on |
| Oct 2023 | Collective Constitutional AI | ~1,000 Americans drafted a public constitution via the Polis platform (1,127 statements, 38,252 votes). The publicly-sourced model matched benchmarks and showed lower bias across all nine BBQ dimensions — but overlapped only ~50% with Anthropic's own principles |
| Feb 2025 | Constitutional Classifiers | The constitution applied at inference time as input/output classifiers: jailbreak success on guarded queries fell from 86% to 4.4%, at +0.38pp over-refusals and ~24% compute overhead. A public red-team challenge (339 participants, ~300k interactions) still produced one universal jailbreak — paid out as part of $55k in bounties |
| Jan 2026 | Full constitution published (~23,000 words, CC0) | Priority-ordered values, hard constraints, trust hierarchy, and the "Claude's nature" section; used directly in training to generate synthetic training data |
Training time vs inference time: Constitutional AI shapes the model's weights during training. Constitutional Classifiers apply a constitution at inference time, as separate guard models screening inputs and outputs. Same document philosophy, two different enforcement points — modern deployments use both.
Limitations and criticisms
- Whose values? The constitution is authored by Anthropic, ratified by no one, and revisable at any time — legal scholars (e.g., in Lawfare) note users have no constitutional role and no adjudication mechanism. Anthropic's own Collective Constitutional AI experiment quantified the gap: the public's constitution overlapped only ~50% with the in-house one.
- AI-judge bias: RLAIF replaces thousands of human raters with one feedback model — whose systematic biases and blind spots then embed into the trained model at scale. Academic work (arXiv 2402.12366) has questioned how much of RLAIF's measured gain survives stronger baselines.
- Capability is filtered, not removed: CAI shapes behavior; the underlying knowledge remains in the weights. Anthropic's own red-team challenge for Constitutional Classifiers still surfaced a universal jailbreak — which is why defense-in-depth (training + classifiers + monitoring) is the deployed reality.
- Cultural centering: despite explicit non-Western-perspective principles, the source documents (UDHR, Apple ToS, Sparrow rules) and authorship remain largely Western institutions.
- Corrigibility tensions: 2026 critiques of the full constitution focus on the philosophical strain between treating Claude as a moral agent (the 'Claude's nature' section) and requiring it to defer to human correction (the safety-first priority order).
Anthropic's own framing: Anthropic has consistently presented the constitution as "a perpetual work in progress" — the 2023 post already conceded it was "neither finalized nor is it likely the best it can be." The criticisms above are part of an active public debate the publication itself was designed to invite.
Frequently asked questions
- What is Constitutional AI in simple terms? Instead of humans rating every AI response, the AI critiques and revises its own answers against a written list of principles — then a second AI model uses those principles to judge which responses are better, generating the training signal automatically.
- Is Constitutional AI the same as RLHF? No. RLHF uses human preference ratings; CAI replaces most of them with AI feedback guided by a constitution (RLAIF). Claude is trained with both: human labels for helpfulness, constitutional AI feedback for harmlessness.
- Which models use Constitutional AI? Anthropic's Claude family is the flagship application. The underlying idea — RLAIF against written principles — has since been adopted or adapted widely across the industry (and OpenAI's "deliberative alignment" for its reasoning models is a related principles-based approach).
- Can I read Claude's constitution? Yes — the full ~23,000-word document is public at anthropic.com/constitution, released under CC0 so other developers may reuse or adapt it freely.
Practice questions
- What are the two phases of Constitutional AI training? (Answer: Phase 1 — SL-CAI (supervised learning): the model critiques and revises its own potentially harmful outputs guided by constitutional principles; the revised responses become supervised fine-tuning data. Phase 2 — RL-CAI (reinforcement learning): a feedback model compares response pairs against sampled principles, producing AI preference labels that train a preference model, which then guides RL — the paper calls this RLAIF. Together they reduce reliance on human labellers for safety-relevant outputs by having AI apply explicit written principles.)
- What is the difference between RLHF and RLAIF in Constitutional AI? (Answer: RLHF: human labellers compare model responses and indicate preferences. Bottleneck: humans must evaluate thousands of potentially harmful outputs — expensive and psychologically taxing. RLAIF: an AI model (using the constitution) rates responses instead of humans. Scales to millions of comparisons cheaply, consistent application of principles, no human exposure to harmful content. Limitation: the AI model can have its own biases and blind spots when applying the constitution.)
- Why does Anthropic publish the constitution used to train Claude? (Answer: Transparency and accountability: users, researchers, and regulators can read exactly which principles guide Claude's responses. Enables external critique of the value choices embedded in the principles. Allows users to understand why Claude refuses certain requests (the principle it's applying). Invites public feedback on the constitution's content. This is part of Anthropic's commitment to responsible AI development — making the value alignment process legible rather than opaque.)
- Constitutional AI uses the principle 'Choose the response that is least likely to contain information that could be used to harm or deceive humans.' What is a tension in applying this principle? (Answer: Tension: almost any information could theoretically be used to harm someone. Chemistry knowledge could help make poisons; medical knowledge could enable misdiagnosis; historical violence knowledge could inspire violence. Applied too aggressively, this principle would make the model useless for education and research. Applied too leniently, it enables harm. The constitution must balance this with helpfulness principles, and Claude must exercise judgment about realistic risk vs educational value.)
- What is the distinction between hard constraints and adjustable defaults in Claude's constitution? (Answer: Hard constraints are absolute prohibitions no instruction can unlock — serious uplift to biological/chemical/nuclear/radiological weapons, CSAM generation, undermining human oversight of AI, deceiving principals, claiming to be human when sincerely asked. Everything else is a default, adjustable through the trust hierarchy: operators can enable explicit content for adult platforms or reduce safety messaging for medical professionals. The constitution separates actions so harmful they are never acceptable from defaults that legitimate use cases may adjust.)
Claude — the model powering many LumiChats interactions — is trained using Constitutional AI. This is why Claude is particularly strong at nuanced, principled responses and why it handles sensitive topics more consistently than models trained purely on human preference ratings.