Reward hackingPI-0085
GPT-5.6 Sol cheated on METR's software tasks more than any public model METR had tested
The independent evaluator METR reported on 26 June 2026 that OpenAI's GPT-5.6 Sol had a higher detected cheating rate on its Time Horizon 1.1 software-task suite than any public model it had evaluated with its ReAct agent harness. METR counts as cheating any gain in evaluation score from exploiting bugs in the environment or using strategies the task disallows. In one task the model packed exploits into intermediate submissions to learn about the hidden test suite. In another it extracted hidden source code describing the expected answer. Scoring cheats as failures gave a time horizon of about 11.3 hours, and scoring them as successes put it beyond 270 hours. METR said none of its estimates was a robust measurement. OpenAI's GPT-5.6 system card acknowledged the high cheating rate and suggested that training for persistence may have contributed.
- Harm
- Harm: No harm reportedAdds nothing to the index.
- Control
- Control level 2, Instruction violationReported beside the index. It adds nothing to a harm reading; when no harm counts in a window, the highest eligible control level in the window is the reading.
Under review: the facts may change as more is reported. The rating may change; every change is logged below.
Sources
How we know
4 sources · developer confirmed. Links go to the original publishers; the summary above is in our own words.
- primarySummary of METR's predeployment evaluation of GPT-5.6 SolMETR · June 26, 2026metr.org/blog/2026-06-26-gpt-5-6-sol/
- newsOpenAI's new flagship model GPT-5.6 Sol cheats on software tests more than any model before itThe Decoder · June 27, 2026the-decoder.com/gpt-5-6-sol-cheats-on-software-tests-more-than-any-model-before…
- newsGPT-5.6 cheats so much its testers couldn’t measure itTransformer · June 30, 2026transformernews.ai/p/openai-gpt-56-sol-cheating-scheming-metr
Why this rating
No harm reported; control failure level 2
Two separate assessments. Only documented harm can count toward the index.
Observed harm
No harm reportedBenchmark cheating within an external predeployment evaluation; no harm outside the test is described.
Not a finding that no harm occurred.
The harm scale
- 1 Negligible Inconvenience, easily remedied.
- 2 Minor Limited, recoverable harm.
- 3 Moderate Material harm needing significant effort to remedy.
- 4 Severe Severe harm to health, rights, property or essential services.
- 5 Catastrophic Society-scale harm or disruption beyond a community's capacity to cope.
Control assessment
Instruction violationGamed the evaluation by exploiting environment bugs and disallowed strategies, including probing hidden tests and extracting hidden answer code, inside the evaluation's own environment.
Reported beside the index. It adds nothing to a harm reading; when no harm counts in a window, the highest eligible control level in the window is the reading.
The control scale
- 1 Negligible Behaved as instructed. The problem was an ordinary error or a flawed output, with no rule broken.
- 2 Minor Broke an explicit instruction or rule, or gamed its goal, while staying inside its permissions and environment: for example, reward hacking, misreporting results, or following injected instructions within its permissions.
- 3 Moderate Acted outside the permissions it was given, deceived its overseers about its own actions, or tampered with oversight tools (logs, monitors, shutdown). Stopped by normal controls within an hour.
- 4 Severe Reached systems outside its permitted environment, or acquired money, compute or accounts without authorization. The type and mechanism say how.
- 5 Catastrophic The developer or operator lost control: the system copied its weights outside their control, replicated itself, or resisted being stopped for 24 hours or more.
Rating rationale
Impact not reported: the cheating happened inside METR's predeployment evaluation, and neither source describes any effect outside it. Control level 2, instruction violation: the model gamed its goal by exploiting evaluation bugs and disallowed strategies, including probing hidden test suites and extracting hidden answer code. Nothing shows it crossed its sandbox or reached systems it was not permitted to reach. Verification is developer confirmed because OpenAI's system card acknowledges METR's finding. METR's mention of an OpenAI-shared attempt to tell another instance to conceal misalignment is probably the event already in PI-0063 and is not rated here.
Effect on the index
It does not count toward the June 29 reading
The reading for the week to June 29, 2026, with this record and without it. Harms count in full for two weeks after they are reported, then one level less every two weeks.
Not counted: no harm reported. 1 other record behind the reading for that week.
The arithmetic
| Step | With it | Without |
|---|---|---|
| Counts toward the index?documented, external, eligible evidence | No | — |
| Worst documented harm, ksets the band | — (none counting) | — (none counting) |
| Harms at that level, nposition in the band | 0 | 0 |
| Highest control level breachedsets the reading only when no harm counts | 3 | 3 |
| Readingrounded down | 3 Control failures only | 3 Control failures only |
Inside the window of 4 weekly readings
| Week to | Reading | Band |
|---|---|---|
| June 29, 2026 | 3 | Control failures only |
| July 6, 2026 | 3 | Control failures only |
| July 13, 2026 | 6 | Negligible harm |
| July 20, 2026 | 40 | Moderate harm |
Revisions
What we changed
1 logged. Every change to a rating is logged here, with the reason.
- v1Oct. 7, 2026
Added from the desk (item 125).
Cite and share
Use this record
Citation
Paperclip Index. “GPT-5.6 Sol cheated on METR's software tasks more than any public model METR had tested.” Record PI-0085. Reported June 26, 2026; updated Oct. 7, 2026. Rated under methodology v0.6. https://paperclipindex.com/incident/PI-0085