Skip to content

8 curated AI tools for Local Model Runtimes

Local model runtimes download and serve open models on your own machine or server, for offline use, privacy-sensitive work and predictable costs.

More on this category

What these tools are

Local model runtimes download, load and serve open models on your own machine or server, usually behind an OpenAI-compatible API. They suit privacy-sensitive work, offline use, and anyone avoiding per-token billing.

What they usually do

Pull a model with one command, run chat and batch jobs locally, wire the model into an existing app, quantize it to fit your VRAM, and serve concurrent inference requests.

Strengths and limits

Data stays on your hardware, there is no per-token fee, and it works offline. The trade-offs are quality below comparable hosted models, speed set by your GPU, and version headaches between models and drivers.

How pricing works

Most runtimes are open source and free; the cost is hardware and electricity. Products with a polished interface may charge for extras, and enterprise support is usually annual.

How to choose
  • Work out how large a quantized model your VRAM can hold
  • Check format and platform support (GGUF, MLX, CUDA)
  • Confirm the API is OpenAI-compatible so existing code can switch over
  • Look for batching and concurrency, not just single-chat use

Maintained by steven · About our listings →