Updated

llama.cpp

C/C++ library for running LLM and vision-language model inference locally on CPU or GPU with minimal dependencies.

27 open-source projects declare llama.cpp in their code; 1 maker's product we track uses it.

Open sourceSelf-hostable

Profile checked

Ask your AI about this, with this page as the source:ChatGPT ↗Claude ↗Perplexity ↗

How makers use llama.cpp

Makers' products we track use llama.cpp for run models locally (1).

What llama.cpp is used for

Run Models Locally · Inference engine →Full control over builds, quantization and hardware flags, or embedding inference inside your own app.Trade-off: You compile it, convert models and pick settings yourself.
Ollama39Llama21LM Studio4llama.cpp1Jan0Makers' products on each, for run models locally

llama.cpp vs the alternatives makers chose

For run models locally, 1 of the 61 makers' products we know the choice of picked llama.cpp (2%), behind Ollama (64%).

OptionMakers' productsOpen source
Ollama39 64%Yes
Llama21 34%No
LM Studio4 7%No
llama.cpp1 2%Yes

For run models locally: how many of the 61 makers' products we know the run models locally of chose each option.

When to use llama.cpp, and when not

Pick llama.cpp if

Something else fits better if

  • You want a local model behind an API for your code and dev tools in minutes: Ollama.
  • You'd rather browse and compare models in a desktop app before writing code: LM Studio or Jan.
  • You work on a Mac with Apple silicon and want the fastest local inference or fine-tuning there: MLX LM.
  • You need a model family with a permissive license for commercial use: Qwen or gpt-oss or Gemma.
From each decision's guide, written from what makers' products chose and why.

How llama.cpp itself is built

7 tools, from its own code, website and Product Hunt page.

Who uses it

1 makers' products, each linked to the source that shows it, and 27 open-source projects that declare it in their code.

The maker says so 1Declared in code 27How evidence is collected →
LM Studio
LM StudioDesktop app for downloading and running local LLMs with a chat UI and an OpenAI-compatib…

“LM Studio is made possible thanks to the llama.cpp project.”

Run Models LocallyMaker says so · source ↗

Open source: a project that declares llama.cpp as a dependency in its public code — verifiable, but not necessarily a live product.

Alternatives to llama.cpp

All alternatives by situation →

Questions makers ask about llama.cpp

What is llama.cpp used for?

C/C++ library for running LLM and vision-language model inference locally on CPU or GPU with minimal dependencies. Makers' products we track use it for run models locally (1).

What are the alternatives to llama.cpp?

For run models locally, the other options makers' products we track chose most are Ollama (39), Llama (21), LM Studio (4). Open source or self-hostable among them: Ollama, Llama, LM Studio. Each alternative, with the situation it fits, is on the llama.cpp alternatives page.

When should you use llama.cpp?

You ship local inference inside your own app or need settings the apps don't expose (run models locally). Something else fits better if: you want a local model behind an API for your code and dev tools in minutes, pick Ollama; you'd rather browse and compare models in a desktop app before writing code, pick LM Studio or Jan.

Is llama.cpp open source or self-hostable?

Open source, and you can self-host it. source ↗

Can AI coding agents work with llama.cpp?

No llms.txt, official MCP server or CLI found yet.

Who uses llama.cpp?

1 makers' products we track, each with a source, and 27 open-source projects declare it in their code. source ↗