Home / Rank #10
LLM gateways & spend-tracking tools
A gateway (LiteLLM, Portkey, OpenRouter) gives you one API across providers plus the control points every other method needs: caching, routing, fallbacks, budgets, rate limits, and per-request cost logging. Spend-tracking layers (Helicone — now maintenance-only after its Mintlify acquisition — nOps, native provider dashboards) add the visibility.
On its own a gateway saves little; as the enforcement point for caching, routing, and attribution it is how the savings become systematic instead of one-off.
How to do it
- Route all LLM traffic through one gateway; ban direct provider SDK calls in code review.
- Turn on request logging with cost metadata from day one.
- Enable provider fallbacks (resilience) and budget caps (runaway protection).
- Layer caching and routing policies in the gateway rather than per-app.
Frequently asked questions
Which gateway should I pick?
LiteLLM (open-source, self-hosted, 100+ providers) for engineering-led teams; Portkey for managed guardrails and semantic caching; OpenRouter when you want one bill across many model vendors. See our tool reviews for details.
Tools for this method
LiteLLM
The default self-hosted gateway: one OpenAI-compatible proxy across 100+ providers with budgets, caching, routing, fallbacks, and …
Portkey
Managed gateway with the strongest built-in semantic caching, plus guardrails, routing, and cost analytics. The low-ops route to t…
OpenRouter
One API and one bill across hundreds of models from every major lab — the fastest way to arbitrage the model price war and A/B che…
Helicone
Long the default open-source LLM cost dashboard. Acquired by Mintlify in March 2026 and now maintenance-only — existing deployment…