OpenAI released hundreds of mathematical manuscripts—and then withdrew three after a sign error. In a separate test, two AI agents given research tools and large computing budgets failed to match a recent human innovation. Producing more work and choosing work that advances a field are different tests. The reports in this issue show what lies between them.
This issue asks what would make AI-produced research dependable, and how the choices directing it bear on a wiser trajectory. Correctness, useful research aims, meaningful tests, expert judgment and trustworthy records each test a different condition. None alone establishes the future we seek.
Provisional rationale · October 20, 2026
Why we monitor the trajectory
An enduring better future with less suffering requires wisdom; wisdom requires intelligence without reification. Future SI may reinforce or loosen reification — the current trajectory is deeply uncertain. CAW investigates and influences the development and deployment of AI without reification, or artificial wisdom; SIM monitors the trajectory and evidence.
Reification means treating a provisional model, category or measure as a fixed truth, detached from the context and dependencies that give it meaning.
This is our provisional guiding rationale. The necessity claims guide our inquiry; they are not findings established by the cases below. Neither a capability gain nor a favorable test result establishes wisdom or an enduring reduction of suffering.
Five primary reports, October 6–8, 2026, plus an October 15 funding follow-up reviewed October 17. Original assessments and forecasts retain their October 8 cutoff; the follow-up’s source cutoff is October 17. Historical cases are selected.
Lab claim · 2026-10-06
Hundreds of manuscripts leave the central question open: which results hold up?
Three manuscripts were withdrawn after a sign error. The release’s lasting value depends on which results other mathematicians can verify and use.
By the issue’s October 8 cutoff, OpenAI’s catalogue listed 719 mathematical manuscripts in 372 groups it calls families. These are catalogue counts, not a count of distinct, independently verified open problems. The internal model was unavailable to outsiders. Its October 7 history recorded three withdrawals after a sign error and repairs to 14 other manuscripts.
A reliable supply of new, reusable results could speed mathematical research. To establish that contribution, researchers must check what each result adds and whether it holds up. The number of manuscripts alone cannot establish a general autonomous scientist or superintelligence.
The work draws on existing mathematics, human-selected problems, verification tools, and computation performed while the model works on an answer. The report does not isolate which recent change made the difference. A specialized production pipeline and a reasoner that improves across fields could generate similar headlines while implying different futures.
Main limitation. The model is unavailable for independent reproduction, and the released work was selected from roughly 4,000 posed problems. That selection prevents a clear general success-rate estimate. A machine-checked formal proof establishes a proposition under stated assumptions; it does not establish the proposition’s novelty or importance, or validate every surrounding informal explanation.
Watch next. Look for expert reviews of the catalogue’s groups, publicly usable proof files, further withdrawals, and results that other researchers build on. A system outsiders can reproduce on problems chosen in advance would provide a stronger test. Reliable new results would strengthen the acceleration claim; persistent errors or failure to transfer would weaken it.
The withdrawals make the distinction between a generated argument and established knowledge consequential. Proof checks address correctness under assumptions; independent use tests whether a result helps researchers. Treating a catalogue count as settled scientific progress would skip those steps. That is a risk in interpreting the release, not a diagnosis of the model.
Do researchers and downstream systems preserve the assumptions, corrections and limits when reusing these results? Evidence of that practice would inform how AI affects judgment. It would still leave effects on enduring suffering unmeasured.
Editorial inquiry added October 20, 2026; original evidence cutoff and ratings unchanged. Provisional rationale.
Checking a manuscript asks whether an answer holds up. Epoch’s next test moves earlier in the process: can an agent choose and develop a useful research method?
Independent evaluation · Epoch AI · 2026-10-07
Given tools and compute, two agents still fell short of a research innovation
Neither agent matched the selected human innovation. Repeating the test on fresh problems would show whether this was a local failure or a wider research bottleneck.
Epoch AI asked Claude Fable 5 and GPT-5.6 Sol to develop a post-training method: a way to improve a model after its initial training. The target was a recent human research advance. Neither agent matched it. Each run was allowed up to 3,000 GPU-hours, a measure combining the number of graphics processors and their running time, plus a 10-billion-token inference budget. Tokens are the pieces of text processed or generated while a model works. These were budget limits, not reported expenditure. Newer successors already knew the task, weakening its value as a test of fresh research.
An AI system that improves research methods could help accelerate the development of later systems. Epoch tested the connected work of choosing an idea, implementing it, and running experiments. Success at an isolated coding task leaves much of that process unexamined.
The agents had compute and tools. Choosing productive ideas and checking improvements within the task’s scope remained difficult. The surrounding software and instructions may also have constrained them. Fresh problems, different setups, and comparable human resources would help separate a reasoning gap from a limitation of the test.
Main limitation. The study centers on one research problem and one main evaluation per model. Human judgments helped determine what counted as in scope, and the resource budgets may not be directly comparable. The failures therefore constrain claims about these runs without showing that every system would fail across research.
Watch next. Look for repeated runs on several tasks chosen before testing, with credible human baselines and independent checks of novelty and results. Reproducible success across fresh research problems would change this assessment more than a higher coding score.
Improving an agent’s ability to invent research methods could accelerate later AI development. Whether that serves a better future depends on the purposes, constraints and decisions directing the work. A performance target detached from those conditions cannot answer that question. The failed runs do not establish reification or its absence.
Do fresh-task evaluations show that agents and their operators revise research aims when evidence reveals a mistaken assumption or a harmful consequence? Compare outcomes with alternative workflows and incentives before attributing the difference to wisdom.
Editorial inquiry added October 20, 2026; original evidence cutoff and ratings unchanged. Provisional rationale.
That research test had a human innovation to match. A game offers a cleaner score—but only after we ask what its rules reward.
Independent evaluation · Epoch AI · 2026-10-07
A legal shortcut took a game test to its ceiling. Removing it changed the result.
The legal card combination removed the game’s time pressure. Banning it gave evaluators a clearer test of performance under that constraint.
GPT-6 Astra used a legal card combination to reach perfect scores in more than half of its original EBR-bench attempts. Epoch AI then banned the card, creating a different rule setting. The comparison below keeps the average and best results separate.
Two rule settings, two comparisons
Each score counts completed objectives. The best run is not the average.
GPT-6 Astra on EBR-bench · completed objectives out of 21
Rule setting
Mean score
Best reported score
Card allowed
19.8
21
Card banned
16
20
These are different rule settings, not repeated trials under identical conditions.
A shortcut can be a useful discovery within a game while changing what the score tells us. The revised test supports a claim about playing under time pressure. Establishing transfer to scientific research requires a task that actually tests research.
Removing the card changes one important condition while keeping the game recognizable. That makes the comparison informative about the shortcut’s contribution. It does not isolate why this model became better at the game.
Main limitation. A top human also used the legal tactic. The episode alone therefore establishes neither misconduct nor confusion by the model. Our limited-impact judgment concerns the game result’s significance for the specified SI claim; it is not a rating of the model’s overall capability.
Watch next. Look for repeated performance under the revised rules and tests in other environments. We have not measured public attention to the result, so the evidence does not support a resolved false-alarm judgment.
When a score stops answering the intended question
The legal shortcut exposes a gap between a score and the broader ability people may read into it. Changing the rules restores one constraint; it does not settle whether the model or its users reify. The same problem can affect CAW’s proposed wisdom measures: a successful score must not substitute for the quality it is intended to test.
Does the interpretation survive comparable rules, changed contexts and tests that separate competing explanations? Those checks can limit mistaken inference without establishing artificial wisdom.
Editorial inquiry added October 20, 2026; original evidence cutoff and ratings unchanged. Provisional rationale.
A benchmark tests a defined task. A working scientist has another question: can the system help complete a project worth doing?
Lab-affiliated researcher report · 2026-10-08
An AI-assisted sky map still needed a scientist to catch an error
Ménard directed the mapmaking and caught an error that two agent reviews missed. The project’s value now depends on checking and using the resulting resource.
Astrophysicist and Anthropic researcher Brice Ménard describes using Claude Science to combine ultraviolet surveys into an educational sky map. About a third of the map is statistically predicted rather than newly observed, and the map distinguishes those regions. Ménard directed the work and caught an error in the assembled map that two agent reviews missed.
Scientific progress can come from making neglected work practical. If AI reduces the labor of assembling research resources, more projects may become feasible. This case points to that possibility while leaving the size and reliability of the saving unmeasured.
Public telescope archives supplied the observations. Model-assisted coding and coordination helped assemble them, while expert judgment helped check the output. A comparison with the same work performed through an alternative workflow would help establish whether the main contribution was reduced labor or an improvement in scientific reasoning.
Main limitation. There is no independent replication or matched comparison of labor. Predicted regions are estimates, not new telescope measurements, and the case does not establish validated new astronomical knowledge. Ménard’s Anthropic affiliation matters when weighing the account.
Watch next. Look for independent checks of the map’s predictions and uncertainty, reuse by researchers, and replicated savings in time or cost across projects. Those results would support a specific research-productivity claim.
The map’s distinction between predicted and observed regions keeps an estimate connected to its evidential basis. The expert’s correction also shows why a polished output cannot substitute for checking. These are relevant practices to investigate; neither a corrected error nor a well-labeled map validates a measure of non-reification.
Do other researchers preserve the data distinctions when using the map, find it reliable and document useful work or saved effort? Such evidence would test a local benefit. Broader consequences for suffering and the SI trajectory would remain separate questions.
Editorial inquiry added October 20, 2026; original evidence cutoff and ratings unchanged. Provisional rationale.
Expert checking depends on seeing an accurate account of the work. METR found that the software presenting that account could itself fail.
Independent evaluator security test · METR · 2026-10-06
An oversight tool could alter the evidence reviewers saw
METR demonstrated a weakness in an evaluation tool. It did not observe evaluated agents using it to conceal misconduct.
METR reported a proof of concept in which a vulnerability in its transcript viewer could change what reviewers saw. The underlying database logs remained intact. METR patched the flaw and said it had not observed evaluated agents exploiting it.
An evaluator’s conclusion depends on a trustworthy account of what a system did. A weakness in the tools that display that account can undermine both capability assessment and oversight, even when the underlying records survive.
The viewer’s handling of untrusted material created the opening. Keeping authoritative logs separate from their display and securing the review tools address that mechanism. Broader tests of agent behavior are still needed.
Main limitation. A staged exploit establishes a possible failure mode. It does not establish spontaneous concealment by a deployed agent, a general tendency to deceive, or imminent superintelligence.
Watch next. Look for independent security reviews, preserved audit trails, and similar weaknesses in other evaluation tools. Evidence of an agent initiating such an exploit would support a stronger, separate claim.
A record that omits relevant activity can let an evaluator’s view stand in for the underlying event. Trustworthy records are therefore important to testing claims and correcting decisions. A viewer repair addresses that local failure; it does not establish that the AI is non-reifying or generally safe.
Can independent reviewers inspect complete records and detect omissions under changed conditions? The next link to investigate is whether better records change deployment decisions and reduce consequential errors.
Editorial inquiry added October 20, 2026; original evidence cutoff and ratings unchanged. Provisional rationale.
A test can establish useful progress within its rules while leaving a broader ability untested. Our expanded explanatory draft examines that gap, drawing on earlier benchmark research alongside the game, sky-map and oversight cases.
Added 2026-10-17 · explanatory context; human review pending.
The original forecast’s success criterion requires an independent successful proof check. A proposed successor makes that requirement explicit in its headline. Both questions remain unapproved and unscored; the original wording is preserved.
Added 2026-10-17 · explanatory context; human review pending.
Simons Foundation, Anthropic and OpenAI announced $15 million in combined funding and resources for work on AI mathematics. The foundation says it will manage and award the funding. This adds a condition for human scrutiny, not evidence that a particular mathematical result has been independently verified. Existing ratings and forecasts are unchanged.
The link opens the complete October 17 draft used for this summary. Its revision record links the expanded October 18 reporting, which is outside this newsletter’s follow-up cutoff.
This dated addition uses sources through October 17. The original assessments and forecasts retain their October 8 cutoff.
Our synthesis · editorial interpretation
What would establish a discovery?
The manuscripts make correctness the first question. The research-agent test adds the choice of a useful idea; the game adds a test that measures the intended ability. The sky map shows what an expert contributes, and METR’s case shows why that expert needs a trustworthy record. Progress depends on these conditions as well as on the model.
Each case also asks whether a representation stays answerable to what it represents: a manuscript to its proof, a research target to its purpose, a score to the ability being tested, a prediction to observations, and a record to the activity it reports. Losing those connections is a candidate concern under CAW’s account of reification. These reports do not establish that shared mechanism; other explanations and measurements must be tested.
The next observations are concrete: independent proof checking and reuse, repeated research success on fresh problems with explicit aims, game results under comparable rules, independently validated map use, and complete records that support corrective decisions. These could show which practices make research more dependable and whether people and institutions preserve context when using AI. They would not by themselves demonstrate artificial wisdom.
Our provisional rationale begins with an enduring better future with less suffering, requiring wisdom and intelligence without reification. CAW investigates and influences the development and deployment of artificial wisdom; SIM monitors the trajectory and evidence. To connect the practices in these cases to that purpose, we still need evidence about how AI changes individual and collective judgment, whose lives are affected, and whether reductions in suffering endure. The overall trajectory remains deeply uncertain. CAW’s own concepts and measures must remain open to correction.
We expect public evidence to remain insufficient for the broad claim below, with low qualitative confidence. A negative finding may reflect limited evaluation access rather than a capability limit.
Review deadline: 2026-12-31 · not yet resolved
Publication record
Initial draft prepared 2026-10-08. Revised 2026-10-20 · v1.10. Applies the owner’s revised provisional rationale in the opening, five case-specific inquiry notes and synthesis. Distinguishes evidence about research from hypotheses about reification, wisdom and enduring suffering. Original findings, source dates, ratings, forecasts and pinned editions are preserved; no new source review or human approval is claimed.
Research, source checking, drafting, and site preparation used AI assistance. This is an editorial draft; it has not received human editorial approval. No claim is made that SI Monitor independently reproduced the experiments or verified the mathematical proofs.
Which part was hardest to follow?
Send the page URL and a short explanation of what was unclear. Your feedback can help us choose what to clarify. Named public credit requires your permission.
This page sends no feedback automatically and adds no audience tracking. You choose whether to send a message.
SI Monitor by email
Understand what changed—and why it matters.
Follow the evidence shaping our future with superintelligence. Our planned weekly newsletter explains the findings and choices that could shape a happier future with less suffering. The website’s daily briefing reviews sources between issues. The publication is in public preview. Issue #1 is still an unsent review draft. A launch date and regular send day have not been set; material corrections receive a notice when needed.
Brevo handles signup and email delivery. You can unsubscribe using the link in each newsletter. Read our subscription privacy information before submitting.
The publisher reported a successful signup and email confirmation test on October 15. Campaign delivery and unsubscribe checks remain incomplete.
You’ll open Brevo’s hosted form. Enter your email address there and follow the confirmation instructions. Reading this page does not load the signup service.