LlamaIndex
LlamaIndex builds document parsing and extraction software plus open-source tools for AI applications, helping teams turn enterprise documents into usable data for AI agents and workflows.

Groq is an AI inference cloud for developers and enterprises that need low-latency model responses at scale, delivered through custom LPU hardware and a tiered platform spanning bare metal, inference, and enterprise controls.
Groq is an AI inference cloud for developers and enterprises that need fast model responses in production, built around custom LPU hardware and a tiered platform that separates infrastructure, inference, and enterprise controls.
Groq has concentrated on inference since 2016 rather than splitting attention between training and serving. Its Language Processing Unit, or LPU, is the company's own silicon, and the platform pairs it with an integrated stack delivered through GroqCloud. The pitch rests on a simple observation: training creates a model's potential, but inference is where that potential turns into delivered value. As more applications lean on AI, the volume of inference requests climbs, and latency becomes the constraint that shapes user experience.
The platform splits into three layers. GroqMetal supplies dedicated bare-metal infrastructure tuned for speed and reliability, leaving configuration in the customer's hands. GroqCore adds a tested inference stack that converts dedicated capacity into production-ready performance without requiring deep infrastructure expertise. GroqAssured layers enterprise-grade governance, auditability, and control on top, aimed at organizations that need scale without loosening oversight. Each tier includes the underlying hardware layers, so the choice comes down to how much management and control a team wants.
Groq publishes specific figures for its rack design: 256 LPUs per rack, 40 PB/s of SRAM bandwidth, 128 GB of on-chip SRAM per rack, 315 PFLOPS of FP8 inference compute, and a target of 1,000 tokens per second per user. The company also describes LPX, a rack design that connects 256 next-generation LPU accelerators to NVIDIA's Vera Rubin platform for low-latency, large-context inference. That configuration is framed as forthcoming rather than currently deployed, which matters for teams planning around it. Groq says it is building hundreds of megawatts of capacity.
Groq reports operating 13 data centers across four continents. Named locations include Liberty Lake, St. Paul, Dallas, and Houston in the United States; Kamloops, Vaudreuil-Dorion, and Calgary in Canada; Vantaa, Finland and London in the United Kingdom and Europe; Sydney, Australia; and Riyadh, Saudi Arabia. For latency-sensitive applications, the spread of regions gives teams options for placing workloads closer to end users.
The API covers more ground than text alone. Documented endpoints include text generation, speech-to-text, text-to-speech, OCR and image recognition, reasoning, content moderation, structured outputs, and prompt caching. Tool use extends to built-in code execution, browser search on select models, and remote tools through MCP connectors. An OpenAI-compatible base URL at https://api.groq.com/openai/v1 lets existing OpenAI-style integrations point at Groq with limited code changes. Service tiers for performance, flex, and batch processing let teams match cost and latency to each workload, and LoRA inference supports fine-tuned adapters.
The console includes spend limits, projects, and model permissions, alongside Prometheus metrics for monitoring and a published production-readiness checklist. Documentation also covers security onboarding and data handling. GroqAssured extends this with governance and auditability for enterprises that need a formal control layer. The company reports more than 1.8 million developers using the platform, and it closed a $350 million Series A round.
Groq makes the most sense for teams that treat inference as the core of their product and want low latency without assembling their own serving stack. The OpenAI-compatible API and broad endpoint coverage lower the cost of trying it, while the tiered structure gives larger organizations a path from managed inference to governed dedicated capacity. Teams that also need to train models will have to look elsewhere, and anyone counting on the LPX configuration should confirm availability before building a roadmap around it.
Subscription pricing is not listed.
Features, pricing and availability may change. Always verify details on the official website. Maintained by steven · Last updated Oct 16, 2026 →
LlamaIndex builds document parsing and extraction software plus open-source tools for AI applications, helping teams turn enterprise documents into usable data for AI agents and workflows.
Flowith is an agentic AI workspace that combines a whiteboard-style canvas, access to 40+ image and video models, and mini apps for design, e-commerce, and social content.
Fine-tune open models faster with far less memory
Jan is an open-source desktop chat application that lets users run open models on their own hardware or connect to hosted providers like OpenAI, Anthropic, and Google. It targets privacy-conscious individuals and teams who want a friendlier local LLM experience than command-line tooling offers.