SuperintelligenceMONITOR
Published by Center for Artificial WisdomLatest newsletter: Issue #1 · draft v1.11

Evaluation validity · current assessment

A legal shortcut took a game test to its ceiling. Removing it changed the result.

The legal card combination removed the game’s time pressure. Banning it gave evaluators a clearer test of performance under that constraint.

Copy clarified October 18, 2026 · evidence basis and ratings unchanged · Earlier complete wording.

Achievement and evidence

GPT-6 Astra used a legal card combination to reach perfect scores in more than half of its original EBR-bench attempts. Epoch AI then banned the card, creating a different rule setting. The comparison below keeps the average and best results separate.

What it changes. A shortcut can be a useful discovery within a game while changing what the score tells us. The revised test supports a claim about playing under time pressure. Establishing transfer to scientific research requires a task that actually tests research.

Main limitation. A top human also used the legal tactic. The episode alone therefore establishes neither misconduct nor confusion by the model. Our limited-impact judgment concerns the game result’s significance for the specified SI claim; it is not a rating of the model’s overall capability.

What made it possible?

Removing the card changes one important condition while keeping the game recognizable. That makes the comparison informative about the shortcut’s contribution. It does not isolate why this model became better at the game.

This causal account is an editorial interpretation. Qualitative contributions are not measured percentages.

Keeping a measure tied to its purpose

When a legal move changes what a test measures, its rules and interpretation need review. How will systems and their evaluators notice when a successful score no longer supports the purpose it was meant to represent? The reported shortcut alone does not show that the model confused a proxy with a purpose. It gives us a concrete reason to examine how judgments change after a measurement problem is found.

Further editorial question added 2026-10-08. The assessment and ratings are unchanged.

What would change our mind?

Look for repeated performance under the revised rules and tests in other environments. We have not measured public attention to the result, so the evidence does not support a resolved false-alarm judgment.

Next review. A revised benchmark report or cross-environment study; otherwise December 31, 2026.

When a score stops answering the intended question

The legal shortcut exposes a gap between a score and the broader ability people may read into it. Changing the rules restores one constraint; it does not settle whether the model or its users reify. The same problem can affect CAW’s proposed wisdom measures: a successful score must not substitute for the quality it is intended to test.

Does the interpretation survive comparable rules, changed contexts and tests that separate competing explanations? Those checks can limit mistaken inference without establishing artificial wisdom.

Editorial inquiry added October 20, 2026; original evidence cutoff and ratings unchanged. Provisional rationale.

How we rate this claim

Claim under review. A perfect EBR-bench score establishes general learning or autonomous scientific discovery.

Evidence type: Independent evaluation · Epoch AI · source date: 2026-10-07 · status: Unresolved

Impact horizon. Current evidential scope; reassess as transfer evidence emerges.

Impact
Limited
Evidence
Supported
Attention
Unassessed
What these ratings mean

Assessment history

Clarified 2026-10-17. Prose rewritten for clarity; evidence cutoff, claim, ratings, and judgment unchanged. Read earlier and replacement wording.

Clarified 2026-10-18. Copy clarification under Publishing Guidelines v1.5 and Website Guidelines v0.6. Dates, evidence basis, ratings and claim remain fixed; complete earlier wording is preserved in Issue #1 v1.7. Read earlier and replacement wording.

Initial assessment · 2026-10-08. No earlier SI Monitor assessment is claimed.

Assessment in the initial Issue #1 draft

Real game-playing progress; the claim that a perfect score establishes general scientific discovery is unsupported.

Impact
Limited
Evidence
Supported
Attention
Unassessed
What these ratings mean

A top human used the same legal tactic. Old and revised configurations need separate comparisons. Limited SI impact refers to this game result, not the model overall.

Read the assessment in Issue #1

Corrections can be sent through the publisher’s contact page. Include this page URL and the supporting evidence.