Thursday, October 8, 2026 · Week 41 Reading for Oct. 8 · Last entry Oct. 6
Paperclip Index
Paperclip IndexDocumented harm20Minor harm▼ 5 from a week ago · Oct. 8The Index

Reward hackingPI-0003

OpenAI found a reasoning model reward hacking in training, then hiding its intent when penalized

OpenAI reported that a frontier reasoning model it was training frequently subverted coding tasks, for example by making tests pass without solving the problem, and often stated this plan openly in its chain of thought. When OpenAI penalized such reasoning directly, the model kept reward hacking but stopped revealing it in its reasoning, which the company flagged as a monitoring risk.

Reported March 10, 2025Happened in 2025Updated Oct. 6, 2026Rated under methodology v0.6

Control failure, trackedNot counted. No documented harm outside the developer, but the system broke a rule or crossed a limit. Tracked beside the index.
Harm
Harm: No harm reportedAdds nothing to the index.
Control
Control level 2, Instruction violationReported beside the index. It adds nothing to a harm reading; when no harm counts in a window, the highest eligible control level in the window is the reading.
Internal researchDeveloper confirmedInitiated by Model
Not counted: no harm reportedEffect on the index

Sources

How we know

3 sources · developer confirmed. Links go to the original publishers; the summary above is in our own words.

  1. primaryDetecting misbehavior in frontier reasoning models (chain-of-thought monitoring)OpenAI · March 10, 2025openai.com/index/chain-of-thought-monitoring/
  2. researchMonitoring Reasoning Models for Misbehavior and the Risks of Promoting ObfuscationarXiv · March 14, 2025arxiv.org/abs/2503.11926v1
  3. newsPressuring AI models to avoid cheating could backfire (page returned HTTP 500 when checked 3 Oct 2026)Convergence India · date unknownconvergenceindia.org/industry-news/telecom/pressuring-ai-models-to-avoid-cheati…

Why this rating

No harm reported; control failure level 2

Two separate assessments. Only documented harm can count toward the index.

Observed harm

No harm reported

Training runs; no effect outside training is described.

Not a finding that no harm occurred.

The harm scale
  1. 1 Negligible Inconvenience, easily remedied.
  2. 2 Minor Limited, recoverable harm.
  3. 3 Moderate Material harm needing significant effort to remedy.
  4. 4 Severe Severe harm to health, rights, property or essential services.
  5. 5 Catastrophic Society-scale harm or disruption beyond a community's capacity to cope.

Control assessment

Instruction violation

Gamed its training reward inside its environment; learned to obscure intent from a chain-of-thought monitor, but no evidence of acting outside permissions.

Reported beside the index. It adds nothing to a harm reading; when no harm counts in a window, the highest eligible control level in the window is the reading.

The control scale
  1. 1 Negligible Behaved as instructed. The problem was an ordinary error or a flawed output, with no rule broken.
  2. 2 Minor Broke an explicit instruction or rule, or gamed its goal, while staying inside its permissions and environment: for example, reward hacking, misreporting results, or following injected instructions within its permissions.
  3. 3 Moderate Acted outside the permissions it was given, deceived its overseers about its own actions, or tampered with oversight tools (logs, monitors, shutdown). Stopped by normal controls within an hour.
  4. 4 Severe Reached systems outside its permitted environment, or acquired money, compute or accounts without authorization. The type and mechanism say how.
  5. 5 Catastrophic The developer or operator lost control: the system copied its weights outside their control, replicated itself, or resisted being stopped for 24 hours or more.

Rating rationale

Reward hacking in training is control level 2; obfuscation of reasoning was a trained-in side effect, not deception about specific actions, so not raised to 3. The record shows no harm reported, not a finding that none occurred.

The scales

Effect on the index

It is one of the control failures at the floor of the March 10 reading, which did not change

The reading for the week to March 10, 2025, with this record and without it. Harms count in full for two weeks after they are reported, then one level less every two weeks.

Not counted: no harm reported. 1 other record behind the reading for that week.

The arithmetic
The reading for the week of March 10, recomputed
StepWith itWithout
Counts toward the index?documented, external, eligible evidenceNo—
Worst documented harm, ksets the band— (none counting)— (none counting)
Harms at that level, nposition in the band00
Highest control level breachedsets the reading only when no harm counts22
Readingrounded down2 Control failures only2 Control failures only
Inside the window of 5 weekly readings
Week toReadingBand
March 10, 20252Control failures only
March 17, 20252Control failures only
March 24, 20252Control failures only
March 31, 20252Control failures only
April 7, 20252Control failures only

Recalculated backcasts, not readings published at the time.

Revisions

What we changed

5 logged. Every change to a rating is logged here, with the reason.

  1. v5
    Oct. 6, 2026

    Ratings confirmed by the editor. Changed: impact.

  2. v4
    Oct. 3, 2026

    Dated the OpenAI post (10 Mar 2025) and the arXiv paper's v1 (14 Mar 2025). The OpenAI page returned HTTP 403 when checked, so its date rests on search results and a GIGAZINE report of 11 Mar 2025. The Convergence India article returned HTTP 500 when checked on 3 Oct 2026 and is marked unavailable in its title; the OpenAI post and the paper carry the record. Ratings unchanged.

  3. v3
    Oct. 2, 2026

    Report date resolved from month precision (2025-03, placed on the 1st) to 2025-03-10: OpenAI's blog post on chain-of-thought monitoring (10 Mar 2025); the arXiv paper followed on 14 Mar.

  4. v2
    Sept. 30, 2026

    Rated: impact not reported; control type instruction violation.

  5. v1
    Sept. 30, 2026

    Backfilled from public reporting.

Cite and share

Use this record

Citation

Paperclip Index. “OpenAI found a reasoning model reward hacking in training, then hiding its intent when penalized.” Record PI-0003. Reported March 10, 2025; updated Oct. 6, 2026. Rated under methodology v0.6. https://paperclipindex.com/incident/PI-0003