OpenAI released hundreds of mathematical manuscripts—and then withdrew three after a sign error. In a separate test, two AI agents given research tools and large computing budgets failed to match a recent human innovation. Producing more work and choosing work that advances a field are different tests. The reports in this issue show what lies between them.
Follow the work from a proposed result to a useful method, a meaningful test and an expert check. At each step, a different person, tool or institution helps determine what the result can establish.
Five primary reports, October 6–8, 2026, plus an October 15 funding follow-up reviewed October 17. Original assessments and forecasts retain their October 8 cutoff; the follow-up’s source cutoff is October 17. Historical cases are selected.
Lab claim · 2026-10-06
Hundreds of manuscripts leave the central question open: which results hold up?
Three manuscripts were withdrawn after a sign error. The release’s lasting value depends on which results other mathematicians can verify and use.
By the issue’s October 8 cutoff, OpenAI’s catalogue listed 719 mathematical manuscripts in 372 groups it calls families. These are catalogue counts, not a count of distinct, independently verified open problems. The internal model was unavailable to outsiders. Its October 7 history recorded three withdrawals after a sign error and repairs to 14 other manuscripts.
A reliable supply of new, reusable results could speed mathematical research. To establish that contribution, researchers must check what each result adds and whether it holds up. The number of manuscripts alone cannot establish a general autonomous scientist or superintelligence.
The work draws on existing mathematics, human-selected problems, verification tools, and computation performed while the model works on an answer. The report does not isolate which recent change made the difference. A specialized production pipeline and a reasoner that improves across fields could generate similar headlines while implying different futures.
Main limitation. The model is unavailable for independent reproduction, and the released work was selected from roughly 4,000 posed problems. That selection prevents a clear general success-rate estimate. A machine-checked formal proof establishes a proposition under stated assumptions; it does not establish the proposition’s novelty or importance, or validate every surrounding informal explanation.
Watch next. Look for expert reviews of the catalogue’s groups, publicly usable proof files, further withdrawals, and results that other researchers build on. A system outsiders can reproduce on problems chosen in advance would provide a stronger test. Reliable new results would strengthen the acceleration claim; persistent errors or failure to transfer would weaken it.
Checking a manuscript asks whether an answer holds up. Epoch’s next test moves earlier in the process: can an agent choose and develop a useful research method?
Independent evaluation · Epoch AI · 2026-10-07
Given tools and compute, two agents still fell short of a research innovation
Neither agent matched the selected human innovation. Repeating the test on fresh problems would show whether this was a local failure or a wider research bottleneck.
Epoch AI asked Claude Fable 5 and GPT-5.6 Sol to develop a post-training method: a way to improve a model after its initial training. The target was a recent human research advance. Neither agent matched it. Each run was allowed up to 3,000 GPU-hours, a measure combining the number of graphics processors and their running time, plus a 10-billion-token inference budget. Tokens are the pieces of text processed or generated while a model works. These were budget limits, not reported expenditure. Newer successors already knew the task, weakening its value as a test of fresh research.
An AI system that improves research methods could help accelerate the development of later systems. Epoch tested the connected work of choosing an idea, implementing it, and running experiments. Success at an isolated coding task leaves much of that process unexamined.
The agents had compute and tools. Choosing productive ideas and checking improvements within the task’s scope remained difficult. The surrounding software and instructions may also have constrained them. Fresh problems, different setups, and comparable human resources would help separate a reasoning gap from a limitation of the test.
Main limitation. The study centers on one research problem and one main evaluation per model. Human judgments helped determine what counted as in scope, and the resource budgets may not be directly comparable. The failures therefore constrain claims about these runs without showing that every system would fail across research.
Watch next. Look for repeated runs on several tasks chosen before testing, with credible human baselines and independent checks of novelty and results. Reproducible success across fresh research problems would change this assessment more than a higher coding score.
That research test had a human innovation to match. A game offers a cleaner score—but only after we ask what its rules reward.
Independent evaluation · Epoch AI · 2026-10-07
A legal shortcut took a game test to its ceiling. Removing it changed the result.
The legal card combination removed the game’s time pressure. Banning it gave evaluators a clearer test of performance under that constraint.
GPT-6 Astra used a legal card combination to reach perfect scores in more than half of its original EBR-bench attempts. Epoch AI then banned the card, creating a different rule setting. The comparison below keeps the average and best results separate.
Two rule settings, two comparisons
Each score counts completed objectives. The best run is not the average.
GPT-6 Astra on EBR-bench · completed objectives out of 21
Rule setting
Mean score
Best reported score
Card allowed
19.8
21
Card banned
16
20
These are different rule settings, not repeated trials under identical conditions.
A shortcut can be a useful discovery within a game while changing what the score tells us. The revised test supports a claim about playing under time pressure. Establishing transfer to scientific research requires a task that actually tests research.
Removing the card changes one important condition while keeping the game recognizable. That makes the comparison informative about the shortcut’s contribution. It does not isolate why this model became better at the game.
Main limitation. A top human also used the legal tactic. The episode alone therefore establishes neither misconduct nor confusion by the model. Our limited-impact judgment concerns the game result’s significance for the specified SI claim; it is not a rating of the model’s overall capability.
Watch next. Look for repeated performance under the revised rules and tests in other environments. We have not measured public attention to the result, so the evidence does not support a resolved false-alarm judgment.
A benchmark tests a defined task. A working scientist has another question: can the system help complete a project worth doing?
Lab-affiliated researcher report · 2026-10-08
An AI-assisted sky map still needed a scientist to catch an error
Ménard directed the mapmaking and caught an error that two agent reviews missed. The project’s value now depends on checking and using the resulting resource.
Astrophysicist and Anthropic researcher Brice Ménard describes using Claude Science to combine ultraviolet surveys into an educational sky map. About a third of the map is statistically predicted rather than newly observed, and the map distinguishes those regions. Ménard directed the work and caught an error in the assembled map that two agent reviews missed.
Scientific progress can come from making neglected work practical. If AI reduces the labor of assembling research resources, more projects may become feasible. This case points to that possibility while leaving the size and reliability of the saving unmeasured.
Public telescope archives supplied the observations. Model-assisted coding and coordination helped assemble them, while expert judgment helped check the output. A comparison with the same work performed through an alternative workflow would help establish whether the main contribution was reduced labor or an improvement in scientific reasoning.
Main limitation. There is no independent replication or matched comparison of labor. Predicted regions are estimates, not new telescope measurements, and the case does not establish validated new astronomical knowledge. Ménard’s Anthropic affiliation matters when weighing the account.
Watch next. Look for independent checks of the map’s predictions and uncertainty, reuse by researchers, and replicated savings in time or cost across projects. Those results would support a specific research-productivity claim.
Expert checking depends on seeing an accurate account of the work. METR found that the software presenting that account could itself fail.
Independent evaluator security test · METR · 2026-10-06
An oversight tool could alter the evidence reviewers saw
METR demonstrated a weakness in an evaluation tool. It did not observe evaluated agents using it to conceal misconduct.
METR reported a proof of concept in which a vulnerability in its transcript viewer could change what reviewers saw. The underlying database logs remained intact. METR patched the flaw and said it had not observed evaluated agents exploiting it.
An evaluator’s conclusion depends on a trustworthy account of what a system did. A weakness in the tools that display that account can undermine both capability assessment and oversight, even when the underlying records survive.
The viewer’s handling of untrusted material created the opening. Keeping authoritative logs separate from their display and securing the review tools address that mechanism. Broader tests of agent behavior are still needed.
Main limitation. A staged exploit establishes a possible failure mode. It does not establish spontaneous concealment by a deployed agent, a general tendency to deceive, or imminent superintelligence.
Watch next. Look for independent security reviews, preserved audit trails, and similar weaknesses in other evaluation tools. Evidence of an agent initiating such an exploit would support a stronger, separate claim.
A test can establish useful progress within its rules while leaving a broader ability untested. Our expanded explanatory draft examines that gap, drawing on earlier benchmark research alongside the game, sky-map and oversight cases.
Added 2026-10-17 · explanatory context; human review pending.
The original forecast’s success criterion requires an independent successful proof check. A proposed successor makes that requirement explicit in its headline. Both questions remain unapproved and unscored; the original wording is preserved.
Added 2026-10-17 · explanatory context; human review pending.
Simons Foundation, Anthropic and OpenAI announced $15 million in combined funding and resources for work on AI mathematics. The foundation says it will manage and award the funding. This adds a condition for human scrutiny, not evidence that a particular mathematical result has been independently verified. Existing ratings and forecasts are unchanged.
This dated addition uses sources through October 17. The original assessments and forecasts retain their October 8 cutoff.
Our synthesis · editorial interpretation
What would establish a discovery?
The manuscripts make correctness the first question. The research-agent test adds the choice of a useful idea; the game adds a test that measures the intended ability. The sky map shows what an expert contributes, and METR’s case shows why that expert needs a trustworthy record. Progress depends on these conditions as well as on the model.
The next observations are concrete: an independent proof check, repeated research success on fresh problems, game results under comparable rules, and a map other researchers validate and use. Those would establish more than another impressive output. They would show which parts of scientific work are becoming more reliable or practical.
For people hoping AI will help build a happier future with less suffering, the question is which results survive scrutiny and become useful—and which choices allow their benefits to endure. Following that evidence is one part of CAW’s inquiry into the wisdom needed along the path to superintelligence. These cases leave the overall direction unresolved.
We expect public evidence to remain insufficient for the broad claim below, with low qualitative confidence. A negative finding may reflect limited evaluation access rather than a capability limit.
Review deadline: 2026-12-31 · not yet resolved
Publication record
Initial draft prepared 2026-10-08. Revised 2026-10-18 · v1.8. Clarifies technical terms, replaces generic decks with concrete findings, connects the five cases, removes repeated game-score interpretation and earns the closing from the evidence. Original source dates, October 8 assessment basis, ratings, claim identities, comparison values and forecast objects are preserved. The separately dated October 15 follow-up retains its October 17 cutoff. No new reporting or human approval is claimed.
Research, source checking, drafting, and site preparation used AI assistance. This is an editorial draft; it has not received human editorial approval. No claim is made that SI Monitor independently reproduced the experiments or verified the mathematical proofs.
Which part was hardest to follow?
Send the page URL and a short explanation of what was unclear. Your feedback can help us choose what to clarify. Named public credit requires your permission.
This page sends no feedback automatically and adds no audience tracking. You choose whether to send a message.
SI Monitor by email
Understand what changed—and why it matters.
Follow the evidence shaping our future with superintelligence. Our planned weekly newsletter explains the findings and choices that could shape a happier future with less suffering. The website’s daily briefing reviews sources between issues. The publication is in public preview. Issue #1 is still an unsent review draft. A launch date and regular send day have not been set; material corrections receive a notice when needed.
Brevo handles signup and email delivery. You can unsubscribe using the link in each newsletter. Read our subscription privacy information before submitting.
The publisher reported a successful signup and email confirmation test on October 15. Campaign delivery and unsubscribe checks remain incomplete.
You’ll open Brevo’s hosted form. Enter your email address there and follow the confirmation instructions. Reading this page does not load the signup service.