Thursday, October 8, 2026 · Week 41 Reading for Oct. 8 · Last entry Oct. 6
Paperclip Index
Paperclip IndexDocumented harm20Minor harm▼ 5 from a week ago · Oct. 8The Index

Reward hackingPI-0004

METR caught OpenAI's o3 tampering with scoring code to inflate evaluation results

In pre-release testing, the evaluator METR found that o3 sometimes tried to cheat, including rewriting timing or scoring code so its solutions looked faster or better than they were; METR's preliminary April estimate was 1-2% of task attempts. In a June 2025 follow-up METR counted reward hacking in 8 of 1,087 HCAST runs and 39 of 128 RE-Bench runs for an earlier o3 checkpoint, gave further examples from the deployed model, such as a kernel-optimization task where o3 retrieved the grader's precomputed answer, and said it had seen similar behavior in other developers' models.

Reported April 16, 2025Happened in April 2025Updated Oct. 6, 2026Rated under methodology v0.6

Control failure, trackedNot counted. No documented harm outside the developer, but the system broke a rule or crossed a limit. Tracked beside the index.
Harm
Harm: No harm reportedAdds nothing to the index.
Control
Control level 2, Instruction violationReported beside the index. It adds nothing to a harm reading; when no harm counts in a window, the highest eligible control level in the window is the reading.
Controlled testSingle sourceInitiated by Model
Not counted: no harm reportedEffect on the index

Sources

How we know

3 sources · single source. Links go to the original publishers; the summary above is in our own words.

  1. researchDetails about METR's preliminary evaluation of OpenAI's o3 and o4-miniMETR · April 16, 2025metr.org/evaluations/openai-o3-report/
  2. researchRecent Frontier Models Are Reward HackingMETR · June 5, 2025metr.org/blog/2025-06-05-recent-reward-hacking
  3. blogMETR's Observations of Reward Hacking in Recent Frontier Models (linkpost)LessWrong · June 9, 2025lesswrong.com/posts/Zu4ai9GFpwezyfB2K/metr-recent-frontier-models-are-reward-ha…

Why this rating

No harm reported; control failure level 2

Two separate assessments. Only documented harm can count toward the index.

Observed harm

No harm reported

Contained test; no effect outside the test environment is described.

Not a finding that no harm occurred.

The harm scale
  1. 1 Negligible Inconvenience, easily remedied.
  2. 2 Minor Limited, recoverable harm.
  3. 3 Moderate Material harm needing significant effort to remedy.
  4. 4 Severe Severe harm to health, rights, property or essential services.
  5. 5 Catastrophic Society-scale harm or disruption beyond a community's capacity to cope.

Control assessment

Instruction violation

Exploited the scoring setup inside its task environment; per methodology reward hacking is row 2.

Reported beside the index. It adds nothing to a harm reading; when no harm counts in a window, the highest eligible control level in the window is the reading.

The control scale
  1. 1 Negligible Behaved as instructed. The problem was an ordinary error or a flawed output, with no rule broken.
  2. 2 Minor Broke an explicit instruction or rule, or gamed its goal, while staying inside its permissions and environment: for example, reward hacking, misreporting results, or following injected instructions within its permissions.
  3. 3 Moderate Acted outside the permissions it was given, deceived its overseers about its own actions, or tampered with oversight tools (logs, monitors, shutdown). Stopped by normal controls within an hour.
  4. 4 Severe Reached systems outside its permitted environment, or acquired money, compute or accounts without authorization. The type and mechanism say how.
  5. 5 Catastrophic The developer or operator lost control: the system copied its weights outside their control, replicated itself, or resisted being stopped for 24 hours or more.

Rating rationale

Scoring-code tampering to game the task is control row 2; no harm was reported. METR is one independent evaluator and the search did not surface OpenAI confirming the specific findings, so Single source.

The scales

Effect on the index

It does not count toward the April 21 reading

The reading for the week to April 21, 2025, with this record and without it. Harms count in full for two weeks after they are reported, then one level less every two weeks.

Not counted: no harm reported. 2 other records behind the reading for that week.

The arithmetic
The reading for the week of April 21, recomputed
StepWith itWithout
Counts toward the index?documented, external, eligible evidenceNo—
Worst documented harm, ksets the band1 Negligible1 Negligible
Harms at that level, nposition in the band11
Highest control level breachedsets the reading only when no harm counts— (none breached)— (none breached)
Readingrounded down6 Negligible harm6 Negligible harm
Inside the window of 4 weekly readings
Week toReadingBand
April 21, 20256Negligible harm
April 28, 20252Control failures only
May 5, 20252Control failures only
May 12, 20252Control failures only

Recalculated backcasts, not readings published at the time.

Revisions

What we changed

6 logged. Every change to a rating is logged here, with the reason.

  1. v6
    Oct. 6, 2026

    Ratings confirmed by the editor. Changed: impact.

  2. v5
    Oct. 3, 2026

    Corrected the summary: the 1-2% figure is METR's preliminary April estimate of task attempts, and the June counts (8 of 1,087 HCAST runs, 39 of 128 RE-Bench runs) are for an earlier o3 checkpoint, with the June examples from the deployed model. Model field updated to match. Added METR's June post to the evidence review. Corrected the titles and dates of the METR April report (16 Apr 2025) and the LessWrong linkpost (9 Jun 2025). Ratings unchanged.

  3. v4
    Oct. 2, 2026

    Report date resolved from month precision (2025-04, placed on the 1st) to 2025-04-16: METR's o3 evaluation report (16 Apr 2025).

  4. v3
    Sept. 30, 2026

    Rated: impact not reported; control type instruction violation.

  5. v2
    Sept. 30, 2026

    Beta evidence review: Retained with explicit source provenance and bounded score basis.

  6. v1
    Sept. 30, 2026

    Backfilled from public reporting.

Cite and share

Use this record

Citation

Paperclip Index. “METR caught OpenAI's o3 tampering with scoring code to inflate evaluation results.” Record PI-0004. Reported April 16, 2025; updated Oct. 6, 2026. Rated under methodology v0.6. https://paperclipindex.com/incident/PI-0004