collected.tools.

LLM Gateway

One API for models from over 40 providers

What it is

LLM Gateway is a gateway with an OpenAI-compatible interface that gives access to more than 250 models from over 40 providers. Your existing SDKs stay as they are, you only change the model name, and during outages the gateway reroutes requests to other providers of the same model. Cost, latency and errors are tracked per request, and the core is open source under AGPL-3.0 and can be self-hosted with Docker or Kubernetes.

Why it made the cut
  • OpenAI-compatible, switch models with one string
  • Automatic failover between providers
  • Per-request costs and spending limits

Related entries