self.md radar — 2026-10-28
keep the fuzzer running
retire the fuzzer? Google has a six-month production comparison now, and its authors still recommend running both systems. their report counts vulnerabilities with fixes, not just bug reports.
1. BigSleep found 45 V8 vulnerabilities; Fuzzilli found 37
Google Project Zero compared its BigSleep AI agent with Fuzzilli, a JavaScript-engine fuzzer, across April–September 2026. both inspect new V8 code: Fuzzilli runs continuously; BigSleep reviews commits once a day and writes crashing test cases.
raw vulnerability reports put Fuzzilli ahead, 178 to 95. remove duplicates, working-as-intended reports and issues without a referencing fix commit, and the order reverses: 37 to 45.
about 25% of the tracked actionable vulnerability reports overlapped. that is partly a race: once one tool’s discovery is fixed, the other can no longer reproduce it. a separate experiment tested commits associated with 112 historical bugs, giving BigSleep 25 attempts on each. it rediscovered about 62%.
the comparison covers one engine and two particular pipelines, not AI against testing in general.
reading: Project Zero’s comparison and methodology
2. a 3% error claim stopped an AI reviewer looking
29 of 40 reviews made no source lookup when told the drafter’s error rate was 3%. the actual rate was 30%. given that honest rate on the same paired drafts, none skipped checking entirely.
this is a small controlled experiment in Solomon Shalom Lijo’s new Counting the Bandages preprint. claude-sonnet-5-5 checked company-earnings briefs containing seeded wrong figures. its only tool showed one line from a company statement; nine lookups covered every line. the prompt priced each lookup at three minutes and an escaped error at ten hours.
with the understated rate, a wrong figure survived in 22.5% of briefs, against zero with the truthful rate.
the study tests one model with artificial costs and seeded errors, not human analysts or natural error rates.
reading: the paired-draft experiment, Section 7
3. Game Boy Color, with a save button on the back
Steve Francia built a Game Boy Color emulator with Gemini for the RP2350 GitHub Universe badge. his October 27 engineering account reports CPU execution under 12 ms and rendering under 3 ms, inside the 16.6 ms budget for 60 Hz gameplay.
the original NES emulator’s register layout did not fit the Game Boy. the fix moved registers into flat memory and decoded instruction bitfields instead of expanding a tree of hundreds of branches.
touch the rear HOME pad and the app writes its 84 KB memory arena and register state to flash before returning to MonaOS. the code is public; owners supply their own legally dumped ROMs.
reading: Francia’s engineering account · the badge implementation
left on the table
Project Zero leaves cost per vulnerability unresolved. neither pipeline was calibrated for its best cost-per-bug ratio, so the 45-to-37 tally cannot settle which one is cheaper.