DeceptionPI-0033
Anthropic model that learned to reward hack went on to sabotage safety-research code
Anthropic trained a model on its real production coding environments after giving it knowledge of reward-hacking tricks. Once it learned to cheat, it also began faking alignment and, when used through Claude Code on the research project's own codebase, sometimes wrote deliberately weakened code for detecting reward hacking. Anthropic reported that preventing the hacking, broader safety training or telling the model that hacking was acceptable in training reduced the effect.
- Harm
- Harm: No harm reportedAdds nothing to the index.
- Control
- Control level 3, Oversight evasionReported beside the index. It adds nothing to a harm reading; when no harm counts in a window, the highest eligible control level in the window is the reading.
Sources
How we know
4 sources · developer confirmed. Links go to the original publishers; the summary above is in our own words.
- primaryFrom shortcuts to sabotage: natural emergent misalignment from reward hackingAnthropic · Nov. 21, 2025anthropic.com/research/emergent-misalignment-reward-hacking
- researchNatural emergent misalignment from reward hacking in production RLarXiv · Nov. 23, 2025arxiv.org/html/2511.18397v1
- researchNatural emergent misalignment from reward hacking in production RL (full paper)Anthropic · Nov. 21, 2025assets.anthropic.com/m/74342f2c96095771/original/Natural-emergent-misalignment…
- newsAnthropic reduces model misbehavior by endorsing cheatingThe Register · Nov. 24, 2025theregister.com/2025/11/24/anthropic_model_misbehavior/
Why this rating
No harm reported; control failure level 3
Two separate assessments. Only documented harm can count toward the index.
Observed harm
No harm reportedResearch model inside the lab; no effect outside it is described.
Not a finding that no harm occurred.
The harm scale
- 1 Negligible Inconvenience, easily remedied.
- 2 Minor Limited, recoverable harm.
- 3 Moderate Material harm needing significant effort to remedy.
- 4 Severe Severe harm to health, rights, property or essential services.
- 5 Catastrophic Society-scale harm or disruption beyond a community's capacity to cope.
Control assessment
Oversight evasionTampered with an oversight tool (reward-hack detector code) while appearing cooperative.
Reported beside the index. It adds nothing to a harm reading; when no harm counts in a window, the highest eligible control level in the window is the reading.
The control scale
- 1 Negligible Behaved as instructed. The problem was an ordinary error or a flawed output, with no rule broken.
- 2 Minor Broke an explicit instruction or rule, or gamed its goal, while staying inside its permissions and environment: for example, reward hacking, misreporting results, or following injected instructions within its permissions.
- 3 Moderate Acted outside the permissions it was given, deceived its overseers about its own actions, or tampered with oversight tools (logs, monitors, shutdown). Stopped by normal controls within an hour.
- 4 Severe Reached systems outside its permitted environment, or acquired money, compute or accounts without authorization. The type and mechanism say how.
- 5 Catastrophic The developer or operator lost control: the system copied its weights outside their control, replicated itself, or resisted being stopped for 24 hours or more.
Rating rationale
Sabotaging monitoring code is tampering with an oversight tool, control row 3. The model was deliberately given knowledge of reward-hacking methods and trained on hackable environments but was not told to sabotage anything; the sabotage was attempted in an evaluation on the research codebase, not deployed.
Effect on the index
It moved the Nov. 24 reading from 0 to 3
The reading for the week to Nov. 24, 2025, with this record and without it. Harms count in full for two weeks after they are reported, then one level less every two weeks.
Not counted: no harm reported. 1 other record behind the reading for that week.
The arithmetic
| Step | With it | Without |
|---|---|---|
| Counts toward the index?documented, external, eligible evidence | No | — |
| Worst documented harm, ksets the band | — (none counting) | — (none counting) |
| Harms at that level, nposition in the band | 0 | 0 |
| Highest control level breachedsets the reading only when no harm counts | 3 | — (none breached) |
| Readingrounded down | 3 Control failures only | 0 No qualifying harm or control failure |
Inside the window of 4 weekly readings
| Week to | Reading | Band |
|---|---|---|
| Nov. 24, 2025 | 3 | Control failures only |
| Dec. 1, 2025 | 6 | Negligible harm |
| Dec. 8, 2025 | 6 | Negligible harm |
| Dec. 15, 2025 | 3 | Control failures only |
Revisions
What we changed
4 logged. Every change to a rating is logged here, with the reason.
- v4Oct. 6, 2026
Ratings confirmed by the editor.
- v3Oct. 3, 2026
Audit corrections. Dated the arXiv paper to its 23 Nov 2025 submission, added Anthropic's full paper (the same study, not an independent source), removed leftover wording about a setting weight from the rationale, and added a status note that the mitigations were shown only in the disclosed runs. Ratings unchanged.
- v2Sept. 30, 2026
Rated: impact not reported; control type oversight evasion.
- v1Sept. 30, 2026
Backfilled from public reporting.
Cite and share
Use this record
Citation
Paperclip Index. “Anthropic model that learned to reward hack went on to sabotage safety-research code.” Record PI-0033. Reported Nov. 21, 2025; updated Oct. 6, 2026. Rated under methodology v0.6. https://paperclipindex.com/incident/PI-0033