SuperintelligenceMONITOR
Published by Center for Artificial WisdomLatest newsletter: Issue #1 · draft v1.11

Autonomous research · current assessment

Given tools and compute, two agents still fell short of a research innovation

Neither agent matched the selected human innovation. Repeating the test on fresh problems would show whether this was a local failure or a wider research bottleneck.

Copy clarified October 18, 2026 · evidence basis and ratings unchanged · Earlier complete wording.

Achievement and evidence

Epoch AI asked Claude Fable 5 and GPT-5.6 Sol to develop a post-training method: a way to improve a model after its initial training. The target was a recent human research advance. Neither agent matched it. Each run was allowed up to 3,000 GPU-hours, a measure combining the number of graphics processors and their running time, plus a 10-billion-token inference budget. Tokens are the pieces of text processed or generated while a model works. These were budget limits, not reported expenditure. Newer successors already knew the task, weakening its value as a test of fresh research.

What it changes. An AI system that improves research methods could help accelerate the development of later systems. Epoch tested the connected work of choosing an idea, implementing it, and running experiments. Success at an isolated coding task leaves much of that process unexamined.

Main limitation. The study centers on one research problem and one main evaluation per model. Human judgments helped determine what counted as in scope, and the resource budgets may not be directly comparable. The failures therefore constrain claims about these runs without showing that every system would fail across research.

What made it possible?

The agents had compute and tools. Choosing productive ideas and checking improvements within the task’s scope remained difficult. The surrounding software and instructions may also have constrained them. Fresh problems, different setups, and comparable human resources would help separate a reasoning gap from a limitation of the test.

This causal account is an editorial interpretation. Qualitative contributions are not measured percentages.

What would change our mind?

Look for repeated runs on several tasks chosen before testing, with credible human baselines and independent checks of novelty and results. Reproducible success across fresh research problems would change this assessment more than a higher coding score.

Next review. A refreshed InnovationEval or comparable independent multi-task study; otherwise December 31, 2026.

What directs more capable research?

Improving an agent’s ability to invent research methods could accelerate later AI development. Whether that serves a better future depends on the purposes, constraints and decisions directing the work. A performance target detached from those conditions cannot answer that question. The failed runs do not establish reification or its absence.

Do fresh-task evaluations show that agents and their operators revise research aims when evidence reveals a mistaken assumption or a harmful consequence? Compare outcomes with alternative workflows and incentives before attributing the difference to wisdom.

Editorial inquiry added October 20, 2026; original evidence cutoff and ratings unchanged. Provisional rationale.

How we rate this claim

Claim under review. Current frontier agents can already reproduce consequential algorithmic innovation end to end without substantive human intervention.

Evidence type: Independent evaluation · Epoch AI · source date: 2026-10-07 · status: Unresolved

Impact horizon. Present capability in the evaluated setting; wider transfer remains open.

Impact
Meaningful
Evidence
Supported
Attention
Unassessed
What these ratings mean

Assessment history

Clarified 2026-10-17. Prose rewritten for clarity; evidence cutoff, claim, ratings, and judgment unchanged. Read earlier and replacement wording.

Clarified 2026-10-18. Copy clarification under Publishing Guidelines v1.5 and Website Guidelines v0.6. Dates, evidence basis, ratings and claim remain fixed; complete earlier wording is preserved in Issue #1 v1.7. Read earlier and replacement wording.

Initial assessment · 2026-10-08. No earlier SI Monitor assessment is claimed.

Assessment in the initial Issue #1 draft

Evidence against current end-to-end research autonomy in one demanding setting; a broad capability ceiling is not established.

Impact
Meaningful
Evidence
Supported
Attention
Unassessed
What these ratings mean

One research problem, one main evaluation per model, manual scope judgments, a particular scaffold, and uncertain budget comparability constrain the conclusion. This does not show that all research domains or systems would fail.

Read the assessment in Issue #1

Corrections can be sent through the publisher’s contact page. Include this page URL and the supporting evidence.