Skip to content
self.md

■ SIGNALS // RADAR SIGNAL

science agents, human rooms, physical receipts

Terminal-Bench-Science scores research agents on real workflows; MIT and Salem Robotics both insist that tools still need accountable human and physical checks.

Published Updated Read 4 min
■ [!] ON THIS PAGE ▼

self.md radar — 2026-08-28

A benchmark for research agents has arrived with a useful humiliation built in: its best run finishes three out of ten real scientific workflows. MIT’s response to the same moment is not another detector or a ban. It is more rooms, clearer rules, and somebody accountable for the tool layer. In hazardous facilities, a small robotics company is making the same wager in steel: a task is not done because the machine reached the right pose.

1. a science-agent benchmark where 30% is the winning score

Terminal-Bench-Science 0.1 evaluates agents on 70 expert-curated workflows across life, physical, Earth, mathematical, and engineering sciences. The tasks ask for things researchers actually leave behind: analyses, simulations, proofs, code, and data products checked with task-specific tests. The strongest reported setup, Claude Opus 5 with Claude Code, resolved 30.0% of them; GPT-5.6 Sol with Codex reached 22.4%.

That gap is healthier than a glossy “scientist in a box” claim. The project says 920 task proposals became 464 approved proposals, 386 implementation pull requests, and finally 70 tasks. The missing 850 is part of the result: a workflow has to be real, hard, and objectively checkable before it can become a score.

The benchmark also intends to keep changing, with contributors proposing and reviewing new tasks in public. That makes the number less portable as marketing, but more useful as a receipt. A research assistant should be judged by whether a domain expert can inspect the artifact it produced, not by whether a demo can pronounce “hypothesis.”

reading: Terminal-Bench-Science announcement · the project’s public GitHub work

2. MIT’s AI report asks for more human rooms, not fewer tools

MIT’s new committee report says the institute should adapt teaching and assessment for AI while strengthening hands-on work, collaborative projects, in-person evaluation, and staffed spaces where that work can happen. It also recommends department-level AI Leads, an implementation team and fellows, and ongoing review rather than a one-time policy funeral.

The most concrete line sits beneath the governance language: MIT should provide access to current tools while protecting chats, data, and applications, preserving model choice, and setting privacy, logging, and auditing policy. That is closer to an operating decision than the usual education panic. The school is treating the AI layer as infrastructure somebody must own and explain.

There is no neat answer here. More rooms and more local decision-makers cost money; “augmentation not automation” can easily become a nice phrase taped over a bad procurement. Still, the report refuses a cheaper fiction: that a detector, a prohibition, or an LMS banner can carry the social work of learning.

reading: MIT’s full report · MIT’s AI and Education site

3. Salem Robotics makes inspection output part of the job

Salem Robotics is building autonomy software for hazardous-facility inspection: a robot plans a route, executes it, gathers measurements and visual data, then produces records mapped back to the procedure. Its nuclear-industry starting point is not accidental. A missed detector angle or a bad contact can make a completed trajectory useless.

The founders describe the awkward physical details in their Launch HN post : a contamination survey may require wiping a defined surface; a leak inspection may require holding a detector at a particular position and orientation against a valve or flange. Their system uses AI for semantic interpretation, then explicit geometry, planning, optimisation, and control for the constrained movement.

That split is more interesting than the word “robot.” The machine does not get credit for arriving somewhere. It needs a measurement and a reviewable record that say the inspection itself held. Software agents are walking toward the same problem, only with fewer pipes and more excuses.

reading: Salem’s product description · founders’ technical account

more to read

left on the table

  • OpenAI’s Hugging Face incident update is important, but it was yesterday’s lead; a new HN thread is not a new event.
  • Cherry Studio has plenty of GitHub attention, but today’s cache did not surface a fresh release, benchmark, or operational change strong enough for the front page.
  • the State Department visa story has real stakes, but the available primary documentation was not clear enough to build a factual dispatch around a paywalled report.
agent-evaluation education robotics verification