Ollama vs vLLM vs llama.cpp vs LM Studio: choose the local runtime by job

Updated May 14, 2026. This page was rebuilt as a chooser across Ollama, llama.cpp, LM Studio, and vLLM. It avoids frozen benchmark numbers because local inference speed changes with model, quant, hardware, context length, and batching.

Quick pick: use Ollama for a one-person laptop/dev workflow, llama.cpp when you want direct GGUF/build control, LM Studio when you want a desktop UI with a local server, and vLLM when you are running a shared API service on GPU hardware.

If you already know you want the low-level GGUF path, skip the chooser and use the llama.cpp setup guide .

runtime selector matrix

That is the whole decision. Do not start with “which runtime is fastest?” Start with the job. A fast runtime in the wrong workflow still wastes your afternoon.

choose by machine and workflow

If you are not sure, start with Ollama or LM Studio. Move to llama.cpp when the model file, backend, or flags matter. Move to vLLM when the local model becomes a service other people or apps depend on.

API compatibility is similar, not identical

The local runtime world has mostly converged on OpenAI-shaped endpoints, but “OpenAI-compatible” is not a magic spell. Model names, chat templates, sampling defaults, streaming, embeddings, and tool-call behavior can differ.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1/",  # Ollama: partial OpenAI compatibility
    api_key="ollama",                       # required by client, ignored by Ollama
)

# llama.cpp server -> http://localhost:8080/v1
# LM Studio       -> http://localhost:1234/v1
# vLLM            -> http://localhost:8000/v1

Use the shared client shape to reduce glue code, not to skip testing. A prompt that works against one runtime can still behave differently against another because the model, chat template, and defaults changed underneath it.

when to use Ollama

Use Ollama when the important thing is “I want a model running now.” It is the right default for:

• testing prompts locally;
• wiring a prototype to a local model;
• trying a few model families without managing GGUF files manually;
• building personal tools where one human is the user;
• learning what local inference feels like before you pick a lower-level stack.

Ollama also documents native POST /api/chat and POST /api/generate endpoints, so you do not need to force everything through an OpenAI client if the native API is enough.

Do not choose Ollama just because it is easy if you already know you need reproducible build flags, explicit GGUF files, or a shared GPU service.

when to use llama.cpp

Use llama.cpp when the model file and runtime details matter. It is the direct path for GGUF models, CPU/Metal/CUDA/HIP backends, local server control, and precise flags like context size and GPU layer offload.

Pick it when you want to answer questions like:

• Which quant fits this machine?
• Is Metal/CUDA/HIP actually active?
• Can I run this exact .gguf file on this box?
• What happens if I change context size or GPU offload?
• Can I expose a small local OpenAI-compatible server without a bigger serving framework?

If that is your path, use the llama.cpp setup guide instead of staying on this comparison page.

when to use LM Studio

Use LM Studio when the local model workflow is still human-in-the-loop: browse models, load one, chat, test a prompt, and expose a local endpoint for a small app.

It is especially useful when a terminal-first setup would slow the experiment down. The tradeoff is reproducibility: a GUI workflow is comfortable, but it is not the same as a checked-in runbook for a server.

when to use vLLM

Use vLLM when local inference stops being “my laptop is answering me” and becomes “this model is an API.” Current vLLM docs use vllm serve <model> for online serving and describe an OpenAI-compatible server on port 8000.

vllm serve Qwen/Qwen2.5-1.5B-Instruct --generation-config vllm

The --generation-config vllm flag matters because vLLM can otherwise apply generation defaults from the model repository. That may be exactly what you want, but it should not be a surprise.

Use vLLM when you are prepared to treat the runtime like infrastructure: GPU drivers, Python environments, model repos, chat templates, monitoring, and capacity testing.

migration ladder

mistakes that waste time

Choosing by benchmark headline. A throughput number without model, quant, GPU, context, batch shape, and date is trivia. Benchmark your own workload.

Treating OpenAI compatibility as full parity. The endpoint shape may match. Runtime behavior still depends on model names, chat templates, defaults, and unsupported API features.

Using a GUI workflow as deployment architecture. LM Studio is useful. It is not a production plan by itself.

Using a serving framework for a single-person experiment. vLLM is powerful, but it is unnecessary overhead when you just need a local model to answer a prompt.

Staying in a wrapper after you need the metal. Once you care about GGUF files, backend logs, and GPU offload, go to llama.cpp directly.

source-backed notes

• Ollama documents native API endpoints and OpenAI compatibility with parts of the API .
• llama.cpp documents llama-cli, llama-server, GGUF loading, and the OpenAI-compatible server .
• LM Studio documents OpenAI-compatible local endpoints .
• vLLM documents vllm serve and the OpenAI-compatible server .
• Hugging Face documents GGUF as a model format and discovery filter .

related

• Local LLMs with llama.cpp — build, run, and troubleshoot GGUF models locally
• Model Quantization — why Q4/Q5/Q8 choices change what fits on your machine
• Context Optimization — keep prompts small enough for local models
• Multi-Model Routing — route requests when one model is not enough

Next: Build and verify llama.cpp