Groq

Groq is an AI inference cloud built around its own custom-designed chip, the LPU (Language Processing Unit), engineered specifically to run large language models faster and more predictably than general-purpose GPUs. Where GPUs are parallel processors adapted for AI workloads, the LPU's architecture is built end-to-end around the sequential, memory-bandwidth-heavy nature of transformer inference, letting Groq serve models like Llama 3.3, Mixtral, Gemma, and DeepSeek R1 distills at several hundred to over 1,000 tokens per second, independent benchmarks have repeatedly shown it among the fastest inference providers available. Developers access this hardware through GroqCloud, an OpenAI-compatible API that typically requires only a base URL and API key swap to integrate into an existing codebase, plus a browser-based Playground for testing models before writing any code.

The platform is organized into three composable layers: GroqMetal provides dedicated bare-metal infrastructure, GroqCore turns that capacity into a production-ready inference stack without requiring infrastructure expertise, and GroqAssured layers on enterprise-grade governance, auditability, and access control. Beyond raw model hosting, GroqCloud offers Compound and Compound Mini, agentic systems that give any hosted model built-in web search and code execution without custom orchestration, along with Batch API and prompt caching discounts that can stack to roughly 25% of on-demand pricing for high-volume or asynchronous workloads. Groq currently operates 13 data centers across the US, Canada, Europe, the Middle East, and Australia, and reports serving more than 3 million developers, with customers including the McLaren Formula 1 team, the PGA of America, GPTZero, and Fintool.

Groq only hosts open-source and open-weight models rather than proprietary ones like GPT or Claude, positioning it as an inference speed and cost specialist rather than a model developer. Founded in 2016 by Jonathan Ross, one of the original architects of Google's Tensor Processing Unit, Groq raised roughly $750 million in a September 2025 round at a $6.9 billion valuation before Nvidia announced a landmark non-exclusive licensing deal in December 2025, paying an estimated $20 billion for access to Groq's LPU technology and hiring Ross, President Sunny Madra, and other senior engineers. GroqCloud was explicitly excluded from that transaction and continues operating as an independent cloud business under new CEO Simon Edwards, Groq's former CFO, with the company subsequently announcing a further $650 million fundraise in 2026 to continue scaling its global inference footprint and its next-generation LPX rack, which pairs future LPU accelerators with Nvidia's Vera Rubin platform.

GroqCloud uses pay-as-you-go pricing per million tokens with no seat license or minimum spend. Rates range from roughly $0.05 input / $0.08 output for Llama 3.1 8B Instant up to about $1.00 input / $3.00 output for Kimi K2, with the flagship Llama 3.3 70B Versatile priced at $0.59 input / $0.79 output and GPT-OSS 120B at $0.15 input / $0.60 output. Whisper v3 Turbo transcription is priced at $0.04 per hour of audio. A free tier is available to all registered users with no credit card required, offering access to every model at 30 requests per minute. The Batch API and prompt caching each cut rates by roughly 50%, and can be combined for an effective rate of about 25% of on-demand pricing on eligible workloads. Enterprise pricing, including GroqAssured governance features and dedicated GroqMetal infrastructure, is available by contacting Groq's sales team.