Since early 2024, demand for large-language-model (LLM) services has transitioned from isolated testing to continuous, high-volume use across production environments. This shift is especially visible in how organisations run inference – the process of generating responses using a trained model. Two technical and economic forces explain the change. First, training and inference are operationally distinct: training is a capital-heavy, batch procedure for building models, while inference is a latency-sensitive, repeatable task carried out millions of times per day. These two processes require different infrastructure and economics. 

Second, a new layer of service providers has emerged to handle inference delivery. These providers do not operate their own physical infrastructure. Instead, they purchase processing time from cloud platforms and expose it through simplified access points. This has led to a layered market structure, where value is increasingly defined not by who owns the servers, but by who controls access to models. This structure can be understood across four levels: chip manufacturing, cloud infrastructure, model-serving APIs, and integration platforms. 

Building an LLM begins with training: billions of tokens are cycled through clusters of specialised processors—Nvidia H100, Google TPU v5e, or similar—over days or weeks until the model’s parameters converge. This stage is sequential, capital-intensive, and rarely repeated once a model family is set. 

Inference follows: the finished model is loaded into memory, receives a prompt, and generates a response within tens of milliseconds. Because this step may run millions of times per hour, its economics depend on predictable latency, low per-request cost, and continuous hardware availability. Any delay or cost spike at the inference layer scales directly into user experience and margin. 

To deliver inference at scale, the market has evolved into four functional layers—from silicon production to integration tooling. What matters most is no longer ownership of the servers but control over the access point that routes each request to the right hardware and model. 

Layer	Primary Role	Representative Players
L0 – Silicon	Manufacture AI accelerators	Nvidia (H100), Google (TPU v5e), GroqChip
L1 – Cloud Fabric	Lease accelerated clusters	AWS, Azure, Google Cloud, CoreWeave
L2 – Model APIs	Serve proprietary or open-weight models	OpenAI, Anthropic, Google Gemini
L3 – Integration Hubs	Bundle routing, cost controls, and dev tooling	Perplexity, Together.ai, Replicate

This layered arrangement explains why firms with no physical data centers can still dominate traffic and revenue. By abstracting hardware complexity behind stable pricing and straightforward interfaces, they capture the decisive link between end-user demand and raw compute—leaving infrastructure owners to compete on throughput and price rather than on direct customer relationships. 

Access to high-performance processors remains uneven. Google’s TPU v5e is confined to Google Cloud; Amazon operates one of the world’s largest public fleets of Nvidia H100 chips, yet its pay-as-you-go pricing adds separate data-transfer and token-billing fees that raise total cost. Groq markets a proprietary inference chip delivering sub-10 ms median response time and has deployed points of presence in North America and Europe. CoreWeave, an independent infrastructure vendor, plans more than 600 000 GPUs by 2026 and already supplies capacity to several model-hosting platforms. 

Because no single provider combines low unit cost, short latency, and assured availability everywhere, many organisations distribute inference requests across multiple sources. A prompt might execute on TPUs in Council Bluffs, then on H100s in Frankfurt, then on GroqChips in Amsterdam, depending on local queue length, hardware type, and contract rate. Large clouds still lease the underlying processors, but the decision engine that routes each request now sits one layer higher—inside the model API or integration platform. This routing logic is what enables OpenAI to expand capacity via Google Cloud, Anthropic to mix infrastructure partners, and open-weight providers such as Mistral to serve models through Together.ai or Replicate without owning hardware themselves. Control over placement thus migrates from the cloud fabric to the service interface that brokers it. 

In May 2025 OpenAI began leasing Google TPU v5e capacity, ending a period of exclusive dependence on Microsoft Azure. The step illustrates how even the most established model provider now chooses infrastructure pragmatically, balancing hardware performance and cost. Usage trends collected over the past year confirm the same pattern at scale: demand for Google Cloud inference endpoints, Anthropic’s Claude family, and Groq’s low-latency hardware services is rising, while consumption of Amazon and Microsoft’s native AI platforms has stagnated or fallen. 

The redistribution is driven by three immediate factors: 

1. Specialised processors Exclusive or early access to TPU v5e, large H100 clusters, or GroqChip delivers measurable gains in speed-per-token and price-per-million-tokens. 
2. Simplified interfaces and billing Turnkey APIs quote a single rate for generated output, eliminating separate line items for processing time and data transfer. 
3. Rapid model iteration Frequent releases—Gemini 1.5 Pro, Claude 3, GPT-4o—keep providers visibly ahead, drawing development teams toward the most actively updated ecosystems. 

These elements favour suppliers that combine unique silicon, transparent pricing, and fast upgrade cycles. Even so, open-weight models such as Mistral and Meta’s LLaMA, routed through Together.ai or Replicate, are gaining share in research and lightweight production, broadening the market’s competitive base and reinforcing the shift toward multi-provider routing. Strategic leverage thus migrates to service layers that aggregate model catalogues, direct requests to optimal hardware, and present costs in a single predictable metric, while cloud infrastructure vendors compete chiefly on raw processor supply and regional coverage. 

Conclusion 
Inference provisioning now unfolds along two simultaneous trajectories. 

1 – Single-cloud deployments led by the hyperscalers 
AWS, Azure, and Google each deliver first-party LLM endpoints. Google outpaces the field by uniting exclusive TPU v5e capacity with a broad, high-performance model catalogue, thereby capturing incremental traffic across hardware, platform, and model layers. 

2 – Multi-cloud routing emerging from AI-specific providers 

• Large model suppliers. OpenAI augments its longstanding Azure footprint with Google infrastructure; Anthropic distributes requests across several clouds to optimise price and latency. 
• Integration platforms. Perplexity, Together.ai, Replicate, and similar services lease compute from multiple clouds and expose it through a single interface. Their routing software continuously compares queue depth, regional latency, and quoted cost per token, then directs each query to the first site—TPU v5e in Iowa, H100 clusters in Frankfurt, GroqChips in Helsinki—that satisfies predefined economic and performance thresholds. By doing so, they shift pricing power from physical infrastructure to the coordination layer. 

Physical assets still set the ceiling for throughput, yet strategic influence now rests with the software that assigns, request by request, where computer runs and how usage is billed. Unless one supplier controls both unmatched hardware and the most advanced models, the market will continue to balance between dominant single-cloud services and dynamically routed multi-cloud pathways.