Skip to content
Atlas by Comet Lab
Model layer

AI gateways and LLM routers, compared

A gateway sits between your application and the model providers: one API in front of many models, with keys, spend and fallbacks in one place. It keeps the choice of model a decision rather than a dependency. The main question is whether it runs in your environment or as a hosted service in the data path.

Learn the idea Is ChatGPT an LLM? Models, products, and platforms on Horizon

Open this layer in the explorer

At a glance

Every tool in this layer, with where it fits and the facts that usually decide it.

Tool Where it fits Licence Self-hostable Hosted API Maturity
LiteLLM LLM gateway The right layer when a system uses several model providers and you want routing, keys, and spend controlled in one place. Open source Yes No Established
Bifrost LLM gateway A fit when you self-host a gateway under heavy traffic and per-request overhead has to stay minimal. Open source Yes No Growing
Cloudflare AI Gateway Edge AI gateway A fit for teams already on Cloudflare who want caching, limits, and analytics on model traffic with minimal setup. Proprietary No Yes Growing
OpenRouter Hosted model router Best when you want fast access to many models (to evaluate or switch freely) without setting up an account per provider. Proprietary No Yes Growing
Portkey AI gateway A good fit when you want routing and observability from one layer rather than integrating a gateway and a monitoring tool separately. Open source Yes Yes Growing
Requesty Hosted model router A fit when controlling and forecasting model spend across providers is a priority and a hosted router is acceptable. Proprietary No Yes Growing
Vercel AI Gateway Hosted AI gateway A natural fit for teams building on Vercel and the AI SDK who want unified model access without separate accounts. Proprietary No Yes Growing

How to choose

Start from your situation. Where a line says “our”, it comes from Comet Lab’s own builds.

  1. The gateway should run in your own environment

    LiteLLM is our default when a client wants to self-host; Bifrost suits heavy traffic where per-request overhead must stay minimal.

  2. Trying many models quickly

    Hundreds of models behind one API and one bill. Excellent for evaluation; for production we usually move to a self-hosted gateway.

  3. Routing and observability as one decision

    Routing, caching, guardrails and observability from one layer. If those are already solved separately, LiteLLM is lighter.

  4. You already build on Cloudflare or Vercel

    Cloudflare AI Gateway adds caching, limits and analytics with little setup; Vercel AI Gateway gives unified model access alongside the AI SDK.

  5. Forecasting spend across providers

    Spend controls and automatic failover, as a hosted router.