Ollama
Ollama is a runtime and cloud service for running open AI models on macOS, Windows, and Linux or through hosted endpoints, aimed at developers and teams building with open models.
Local model runtimes download and serve open models on your own machine or server, for offline use, privacy-sensitive work and predictable costs.
Ollama is a runtime and cloud service for running open AI models on macOS, Windows, and Linux or through hosted endpoints, aimed at developers and teams building with open models.
Jan is an open-source desktop chat application that lets users run open models on their own hardware or connect to hosted providers like OpenAI, Anthropic, and Google. It targets privacy-conscious individuals and teams who want a friendlier local LLM experience than command-line tooling offers.
Bionic is LM Studio's agent for work and code, letting users create documents, slides, PDFs, and software with local or frontier open models while keeping voice data on the device.
vLLM is an open-source inference and serving engine for large language models, built for teams that need high throughput and efficient memory use when deploying LLMs.
GPT4All runs open-source language models locally on Windows, macOS and Linux, keeping chat and document Q&A on your own machine.
llamafile packages open LLMs and speech-to-text models into single-file executables that run locally on most operating systems and CPU architectures without installation. It is a Mozilla Builders project, now maintained by Mozilla.ai.
Local model runtimes download, load and serve open models on your own machine or server, usually behind an OpenAI-compatible API. They suit privacy-sensitive work, offline use, and anyone avoiding per-token billing.
Pull a model with one command, run chat and batch jobs locally, wire the model into an existing app, quantize it to fit your VRAM, and serve concurrent inference requests.
Data stays on your hardware, there is no per-token fee, and it works offline. The trade-offs are quality below comparable hosted models, speed set by your GPU, and version headaches between models and drivers.
Most runtimes are open source and free; the cost is hardware and electricity. Products with a polished interface may charge for extras, and enterprise support is usually annual.