Skip to content
self.md

■ SIGNALS // RADAR SIGNAL

the 42 GB local model

Swiftlet streams routed Qwen experts from local storage, bringing an 80B model to Apple hardware without pretending the hardware trade-offs disappeared.

Published Updated Read 2 min
■ [!] ON THIS PAGE ▼

self.md radar — 2026-08-04

a model can be private without being small, but it still has to live somewhere. Swiftlet puts an 80B Qwen container on a Mac’s SSD and leaves only the routed pieces in memory; the 42 GB download is the bit nobody gets to wave away.

1. the model moved into the cupboard

sources:

what happened: Swiftlet is an Apache-licensed Swift and Metal runtime for Qwen’s hybrid MoE models on Apple hardware. Its 80B Qwen3-Next package is listed at 42 GB on disk, 4.3 GB peak RAM, and 4.5–5 tokens per second on an M5 Mac; its 35B package is listed at 18 GB on disk and 2.6 GB peak RAM. The project also reports the 35B model running on an iPhone 17 at roughly 1 token per second, while warning that only about 3B parameters are active per token and factual recall follows that smaller active core.

the collision: This is a real shift in what “local” can mean: the model may answer inside your own device, through a loopback-compatible server, without a hosted inference account in the middle. But the custody story now has a 42 GB footnote, a battery meter, and a model whose sparse routing does not magically grant it 80B-sized recall. Private is not weightless; it is hardware, storage, and a claim you can inspect.

question left open: when an assistant’s intelligence fits inside a personal machine only by treating the SSD as a rotating cast of experts, what must its owner be able to verify before calling it theirs?

left on the table

  • Hermes Studio — a dashboard for an agent runtime is too thin as evidence of a changed operating boundary.
  • AgentGuide — another coding-agent scaffold, dropped rather than pretending this week needs another harness story.
  • “don’t be a meat proxy” — sharp advice, but an argument about relayed output rather than a source-side capability change.
  • Personal AI OS tools — the control-plane map for personal agents, receipts, memory, and tools
  • Best MCP servers — connect files, browsers, memory, search, and workflow tools without turning the stack into soup
  • Agent memory systems — what agents should remember, what belongs in logs, and how to avoid memory sludge
local-ai personal-ai model-inference