AI gateway

An AI gateway is a server layer that sits between applications and multiple AI model APIs, relaying every request while centrally handling authentication, usage control, routing, and auditing.

1 articleLast mentioned

An AI gateway is a server layer placed between applications and multiple AI model APIs that relays every request. It is also called an LLM gateway. It unifies several model providers behind a common API and centrally handles authentication and access control, usage metering, cost management, request logging and auditing, routing and fallback between models, rate limiting, and caching.

It extends the API gateway concept to LLM calls and is used by organizations to apply policies without modifying individual applications when switching providers or combining multiple models. Cloudflare AI Gateway, Portkey, LiteLLM, and Bifrost fall into this category.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Personal AI Agents Are Easy to Switch. What Could Still Hold Users?AI Startups ·

    On one AI gateway, a service that routes requests across many models, open-weights models that anyone can download accounted for 55.8% of token volume from July…

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.