2025 AI Infrastructure Benchmark Report

Insights from analyzing 2 trillion+ tokens, across 90+ regions and 650+ teams in production

What to Expect From This Report:

• Trends shaping AI adoption and LLM provider growth.
• Benchmarks to optimize speed, cost and reliability.
• Strategies to scale production-grade AI systems.

Key Findings

4x

Increase in token counts

Average token count per request has quadrupled, reflecting the shift to complex workflows and advanced orchestration.

30%

Average reduction in costs

Caching reduces LLM costs by 38% and accelerates response times, with hit rates peaking in Q&A workflows.

40%

Teams using multiple LLMs

Adoption jumped from 23% to 40% in 10 months, driving improved uptime and lower latency.

About this data: Insights are based on over 2 trillion aggregated, anonymized tokens processed through Portkey AI's Gateway platform. All data has been transformed and simplified for easier understanding, with actual values being different from represented figures. This report reflects only the traffic passing through Portkey's AI Gateway and should not be taken as a complete representation of any company's or provider's actual market presence, capabilities, or performance. No customer data was accessed, analyzed, or compromised in creating this report; all insights are derived solely from metadata and aggregated usage patterns. For year-over-year comparisons, the sample includes processed data from January through December 2024 across 90+ regions. Company size classifications and other metrics are standardized for reporting clarity. These and other report definitions are subject to change. Some data points were excluded to protect customer privacy, and all geographic data has been processed in compliance with applicable data protection regulations. This report represents only a subset of Portkey's platform activity and should not be considered indicative of Portkey's full customer base, business scale, or market presence.