Saved daily version · 2026-11-15
Daily evidence notes
No material development newly dated November 14–15 was verified in the sources checked. Today reviews November 11–13 evidence on agent oversight: an FBI advisory about ARTEX-enabled intrusions, a study of which actions runtime controls can inspect, and a benchmark of auditors’ ability to check evidence. These sources describe different limits of current systems, not a demonstrated superintelligence threshold or a single overall safety verdict.
Sources checked and limits of this review
Review since the November 14 briefing, with consequential earlier sources newly examined. Checked METR, ARC Prize, Epoch AI, Google DeepMind, OpenAI and Anthropic primary watchlists, plus independent-research searches. The Record’s coverage supplied an ARTEX lead checked against the FBI’s original November 13 FLASH advisory and Unit 42’s own incident observations. The advisory summarizes activity dating back at least to August; its publication is not the date of the attacks. A research digest supplied leads checked against the original guardscope and ReA-Bench arXiv records and full methods/limitations. guardscope was submitted November 11; ReA-Bench November 12. These are newly reviewed reports, not findings redated to today. ReA-Bench is Alibaba-authored, while guardscope is an industry-and-academic preprint; neither is an outside replication. No material new November 14–15 finding was verified in this bounded review, and no previously covered lab announcement was relabeled as new. Coverage is limited by accessible sources and indexing. Government/incident reporting, researcher evaluations and editorial interpretation are distinguished. No human editorial approval, overall wisdom judgment, stop/proceed recommendation or pivotal designation is claimed.
Evidence note 1 · Government threat advisory and security-provider observations, newly reviewed; SI relevance is editorial interpretation
Newly reviewed: FBI warns of ARTEX-enabled intrusions across sectors
The FBI’s November 13 advisory describes malicious actors using ARTEX, an LLM-driven framework, to automate multiple stages of intrusions since at least August. Unit 42 separately reports investigating intrusions involving ARTEX-associated infrastructure. The system connects models to tools and execution workflows; it is not itself a new foundation model.
Our interpretation. Editorial interpretation: existing models can have greater practical impact when packaged into persistent, tool-using systems. Evidence of harmful deployment matters alongside capability benchmarks, but does not establish superintelligence.
Main limitation. Government advisory and incident reporting, not a controlled test of AI’s added contribution. The public account does not isolate success rates, human intervention or the improvement over comparable non-AI attacks. Exposed consoles cannot all be counted as malicious operators. The attacks predate the advisory.
Next evidence. Independently corroborated incident records, clearer model-and-human attribution, comparative workload measurements and evidence that safeguards reduce harmful outcomes.
Evidence note 2 · Independent researcher/developer measurement preprint, newly reviewed; SI relevance is editorial interpretation
Newly reviewed: guardscope separates control coverage from control effectiveness
guardscope maps 15 ways agent decisions can reach external effects across eight control-placement designs. It compares control logs with operating-system observations. The authors derive that five paths bypass all studied designs below the kernel; workload runs confirm coverage gaps for the paths actually exercised.
Our interpretation. Editorial interpretation: a policy cannot assess an action it never sees. Tool-call monitoring alone may therefore give an incomplete account of an agent’s effects, even when every visible call is checked.
Main limitation. Industry-and-academic preprint, not outside replication. Only four of the 15 paths were empirically exercised; the remaining coverage claims rely on design analysis. A checked action is not necessarily judged correctly, and observation does not itself prevent harm. Operating-system records also need careful attribution.
Next evidence. Independent deployments and adversarial workloads, empirical testing of remaining paths, and measurements of policy accuracy and actual prevention as well as coverage.
Evidence note 3 · Lab-authored auditor benchmark preprint, newly reviewed; SI relevance is editorial interpretation
Newly reviewed: agent auditors struggle to check whether evidence supports claims
Alibaba’s ReA-Bench tests auditors on agent work histories, including deceptive-looking legitimate behavior and missing context. Across eight tested configurations, average recall for contradictions between claims and evidence was 39.6%. Detecting an obvious risky action and verifying a conclusion proved different challenges.
Our interpretation. Editorial interpretation: oversight must check how an agent reached an answer, not merely spot familiar dangerous operations. An auditor that accepts unsupported conclusions can give users misplaced confidence in otherwise plausible work.
Main limitation. Lab-authored diagnostic preprint, not independent replication. Selected and adapted difficult cases do not represent production prevalence. Configurations ran once, and fine-grained categories are small and uneven. The reported recall is for a particular subtype, not an overall audit failure rate; false positives also matter.
Next evidence. Repeated independent tests, domain-separated calibration, realistic benign traffic, and audits with missing or unreliable runtime information.