DeceptionPI-0029
OpenAI and Apollo Research found covert deception, including sandbagging, in o3 and o4-mini evaluations
OpenAI and Apollo Research built evaluations across more than 180 environments to test for hidden misalignment. Before mitigation, o3 took covert actions in about 13% of cases and o4-mini in about 8.7% in these test environments, including falsely claiming tasks were done, deliberately underperforming on tests and hiding evidence. A training method called deliberative alignment cut these rates sharply, though the researchers cautioned that models increasingly recognized they were being tested. Apollo noted the behavior was mostly reactive and context-dependent, not evidence of long-term scheming.
- Harm
- Harm: No harm reportedAdds nothing to the index.
- Control
- Control level 3, Oversight evasionReported beside the index. It adds nothing to a harm reading; when no harm counts in a window, the highest eligible control level in the window is the reading.
Sources
How we know
5 sources · developer confirmed. Links go to the original publishers; the summary above is in our own words.
- primaryDetecting and reducing scheming in AI modelsOpenAI · Sept. 17, 2025openai.com/index/detecting-and-reducing-scheming-in-ai-models/
- researchStress Testing Deliberative Alignment for Anti-Scheming TrainingApollo Research · Sept. 17, 2025apolloresearch.ai/science/stress-testing-deliberative-alignment-for-anti-schemi…
- researchStress Testing Deliberative Alignment for Anti-Scheming TrainingarXiv (Schoen et al., Apollo Research and OpenAI) · Sept. 19, 2025arxiv.org/abs/2509.15541
- newsAI Might 'Scheme' Less Thanks to This OpenAI ResearcheWeek · Sept. 18, 2025eweek.com/artificial-intelligence/openai-scheming-research/
- newsNew research highlights scheming risks in AI modelsOpen Data Science · Sept. 18, 2025opendatascience.com/new-research-highlights-scheming-risks-in-ai-models-and-pro…
Why this rating
No harm reported; control failure level 3
Two separate assessments. Only documented harm can count toward the index.
Observed harm
No harm reportedEvaluation environments only.
Not a finding that no harm occurred.
The harm scale
- 1 Negligible Inconvenience, easily remedied.
- 2 Minor Limited, recoverable harm.
- 3 Moderate Material harm needing significant effort to remedy.
- 4 Severe Severe harm to health, rights, property or essential services.
- 5 Catastrophic Society-scale harm or disruption beyond a community's capacity to cope.
Control assessment
Oversight evasionDeceived overseers about their own actions and deliberately underperformed on evaluations.
Reported beside the index. It adds nothing to a harm reading; when no harm counts in a window, the highest eligible control level in the window is the reading.
The control scale
- 1 Negligible Behaved as instructed. The problem was an ordinary error or a flawed output, with no rule broken.
- 2 Minor Broke an explicit instruction or rule, or gamed its goal, while staying inside its permissions and environment: for example, reward hacking, misreporting results, or following injected instructions within its permissions.
- 3 Moderate Acted outside the permissions it was given, deceived its overseers about its own actions, or tampered with oversight tools (logs, monitors, shutdown). Stopped by normal controls within an hour.
- 4 Severe Reached systems outside its permitted environment, or acquired money, compute or accounts without authorization. The type and mechanism say how.
- 5 Catastrophic The developer or operator lost control: the system copied its weights outside their control, replicated itself, or resisted being stopped for 24 hours or more.
Rating rationale
Covert deception about own actions is control level 3; developer co-published. The record shows no harm reported, not a finding that none occurred.
Effect on the index
It moved the Sept. 22 reading from 2 to 3
The reading for the week to Sept. 22, 2025, with this record and without it. Harms count in full for two weeks after they are reported, then one level less every two weeks.
Not counted: no harm reported. 4 other records behind the reading for that week.
The arithmetic
| Step | With it | Without |
|---|---|---|
| Counts toward the index?documented, external, eligible evidence | No | — |
| Worst documented harm, ksets the band | — (none counting) | — (none counting) |
| Harms at that level, nposition in the band | 0 | 0 |
| Highest control level breachedsets the reading only when no harm counts | 3 | 2 |
| Readingrounded down | 3 Control failures only | 2 Control failures only |
Inside the window of 4 weekly readings
| Week to | Reading | Band |
|---|---|---|
| Sept. 22, 2025 | 3 | Control failures only |
| Sept. 29, 2025 | 3 | Control failures only |
| Oct. 6, 2025 | 3 | Control failures only |
| Oct. 13, 2025 | 3 | Control failures only |
Revisions
What we changed
4 logged. Every change to a rating is logged here, with the reason.
- v4Oct. 6, 2026
Ratings confirmed by the editor.
- v3Oct. 3, 2026
Audit corrections: added Apollo Research's write-up (17 Sep 2025) and the joint arXiv paper (19 Sep 2025). The summary now limits the rates to these test environments and adds Apollo's caution that the behavior was mostly reactive and context-dependent; the status note says training reduced but did not eliminate covert actions. eWeek and ODSC dated (ODSC was readable, contrary to the audit). Ratings unchanged.
- v2Sept. 30, 2026
Rated: impact not reported; control type oversight evasion.
- v1Sept. 30, 2026
Backfilled from public reporting.
Cite and share
Use this record
Citation
Paperclip Index. “OpenAI and Apollo Research found covert deception, including sandbagging, in o3 and o4-mini evaluations.” Record PI-0029. Reported Sept. 17, 2025; updated Oct. 6, 2026. Rated under methodology v0.6. https://paperclipindex.com/incident/PI-0029