Skip to content

Groq

Groq is an AI inference cloud for developers and enterprises that need low-latency model responses at scale, delivered through custom LPU hardware and a tiered platform spanning bare metal, inference, and enterprise controls.

Free TierModel APIs
150

Overview

Groq is an AI inference cloud for developers and enterprises that need fast model responses in production, built around custom LPU hardware and a tiered platform that separates infrastructure, inference, and enterprise controls.

An Inference-First Architecture

Groq has concentrated on inference since 2016 rather than splitting attention between training and serving. Its Language Processing Unit, or LPU, is the company's own silicon, and the platform pairs it with an integrated stack delivered through GroqCloud. The pitch rests on a simple observation: training creates a model's potential, but inference is where that potential turns into delivered value. As more applications lean on AI, the volume of inference requests climbs, and latency becomes the constraint that shapes user experience.

Inside the Platform Tiers

The platform splits into three layers. GroqMetal supplies dedicated bare-metal infrastructure tuned for speed and reliability, leaving configuration in the customer's hands. GroqCore adds a tested inference stack that converts dedicated capacity into production-ready performance without requiring deep infrastructure expertise. GroqAssured layers enterprise-grade governance, auditability, and control on top, aimed at organizations that need scale without loosening oversight. Each tier includes the underlying hardware layers, so the choice comes down to how much management and control a team wants.

Hardware and Capacity

Groq publishes specific figures for its rack design: 256 LPUs per rack, 40 PB/s of SRAM bandwidth, 128 GB of on-chip SRAM per rack, 315 PFLOPS of FP8 inference compute, and a target of 1,000 tokens per second per user. The company also describes LPX, a rack design that connects 256 next-generation LPU accelerators to NVIDIA's Vera Rubin platform for low-latency, large-context inference. That configuration is framed as forthcoming rather than currently deployed, which matters for teams planning around it. Groq says it is building hundreds of megawatts of capacity.

Global Footprint

Groq reports operating 13 data centers across four continents. Named locations include Liberty Lake, St. Paul, Dallas, and Houston in the United States; Kamloops, Vaudreuil-Dorion, and Calgary in Canada; Vantaa, Finland and London in the United Kingdom and Europe; Sydney, Australia; and Riyadh, Saudi Arabia. For latency-sensitive applications, the spread of regions gives teams options for placing workloads closer to end users.

Developer Surface

The API covers more ground than text alone. Documented endpoints include text generation, speech-to-text, text-to-speech, OCR and image recognition, reasoning, content moderation, structured outputs, and prompt caching. Tool use extends to built-in code execution, browser search on select models, and remote tools through MCP connectors. An OpenAI-compatible base URL at https://api.groq.com/openai/v1 lets existing OpenAI-style integrations point at Groq with limited code changes. Service tiers for performance, flex, and batch processing let teams match cost and latency to each workload, and LoRA inference supports fine-tuned adapters.

Production and Governance Controls

The console includes spend limits, projects, and model permissions, alongside Prometheus metrics for monitoring and a published production-readiness checklist. Documentation also covers security onboarding and data handling. GroqAssured extends this with governance and auditability for enterprises that need a formal control layer. The company reports more than 1.8 million developers using the platform, and it closed a $350 million Series A round.

Where Groq Fits

Groq makes the most sense for teams that treat inference as the core of their product and want low latency without assembling their own serving stack. The OpenAI-compatible API and broad endpoint coverage lower the cost of trying it, while the tiered structure gives larger organizations a path from managed inference to governed dedicated capacity. Teams that also need to train models will have to look elsewhere, and anyone counting on the LPX configuration should confirm availability before building a roadmap around it.

Core features

  • Text generation, speech-to-text, text-to-speech, OCR, and image recognition endpoints
  • Reasoning, structured outputs, content moderation, and prompt caching
  • Tool use with built-in code execution, browser search, and remote MCP connectors
  • OpenAI-compatible API at https://api.groq.com/openai/v1
  • Service tiers for performance, flex, and batch processing
  • LoRA inference and Prometheus metrics for production monitoring
  • Console controls for spend limits, projects, and model permissions
  • SDK libraries and an integrations catalog for coding tools

Use cases

  1. 1Running low-latency chat and text generation in customer-facing apps
  2. 2Transcribing audio and generating speech with hosted models
  3. 3Extracting text from images and documents through vision endpoints
  4. 4Building agent workflows that call tools, execute code, or search the web
  5. 5Serving batch jobs and latency-tolerant workloads on flex or batch tiers
  6. 6Deploying dedicated capacity with enterprise governance through GroqAssured

Strengths and things to check

Strengths

  • Purpose-built LPU hardware targets inference latency rather than general compute
  • OpenAI-compatible API lowers the cost of switching from existing integrations
  • Broad endpoint coverage spans text, speech, vision, and reasoning
  • Tiered platform lets teams choose bare metal, managed inference, or governed capacity
  • Console tools for spend limits and model permissions support production oversight

Things to check

  • The platform focuses on inference, so training workloads fall outside its scope
  • LPX racks tied to NVIDIA Vera Rubin are described as forthcoming, not generally available
  • Performance depends on model and tier selection, so results vary by workload
  • Enterprise governance features sit in the GroqAssured tier rather than the base offering

Pricing

Subscription pricing is not listed.

Basic information

Website
groq.com
Added
Oct 8, 2026
Platforms
Web
Chinese & English
Not marked
API
Not marked
Self-hosting
Not marked
Best for
Groq suits developers, AI product teams, and enterprises that need fast, production-grade model inference through an API or dedicated capacity rather than model training.
Information updated
Oct 16, 2026
Website status
Checking
Corrections · Privacy · Data requests?
Contact us

Features, pricing and availability may change. Always verify details on the official website. Maintained by steven · Last updated Oct 16, 2026 →

Related tools

All categories

LlamaIndex

Enterprise Grade

LlamaIndex builds document parsing and extraction software plus open-source tools for AI applications, helping teams turn enterprise documents into usable data for AI agents and workflows.

Free TierSOC 2
View details