[!] TOPIC ARCHIVE // #LOCAL-LLM
#Local-Llm
All 8 guides, operator dossiers, and signals tagged with #Local-Llm.
Ollama
ollama runs LLMs on your machine and exposes them as a local API. when to use it, when to skip it, and how it fits with the local stack.
the personal AI infrastructure is real now
sovereignty tools, local AI acceleration, and $2B market validation → the personal AI OS graduated from concept to product category this week
Steve Korshakov builds the AI tools he wants to use
Steve Korshakov's local AI pattern is builder agency: run models near your code, own the capture loop, and turn personal workflows into tools.
Georgi Gerganov made local models feel ordinary
Georgi Gerganov's llama.cpp pattern made local LLMs practical: portable runtimes, GGUF model files, and AI infrastructure that stays close to the user.
Voice-First Note Capture: Whisper to Structured Markdown
Use whisper.cpp for local transcription, then LLM post-processing to convert rambling voice memos into structured notes with headers, bullet points, and action items
Ollama vs vLLM vs llama.cpp vs LM Studio: choose the local runtime by job
Choose a local LLM runtime by workload: Ollama for laptop/dev, llama.cpp for GGUF control, LM Studio for a desktop UI, and vLLM for multi-user serving.
Model Quantization: Running 70B Models on a Laptop
Reduce model precision from 32-bit to 4-bit to run large language models locally. Covers k-quants, GGUF, and choosing the right quantization level.
llama.cpp setup guide: build, run, and troubleshoot GGUF models locally
Build llama.cpp, load a GGUF model, run the CLI or server, and verify the install with one smoke test and troubleshooting table.