iCreatAI News
Skip to content
Hacker News AI· carlosmendes·· 46 min agoAI score58

AI Gateway: Manage your company's AI from one place

AI Gateway: Manage your company's AI from one place

AI summary

AI Gateway is a self-hosted, OpenAI-compatible endpoint that routes requests to your own GPU servers (vLLM, Ollama) first and falls back to providers like OpenAI, Groq and Together when needed. It adds per-user keys, token budgets and model access controls, plus a dashboard tracking tokens and cost per user, key, model and provider, and it does not log or store prompts and responses. Deployment is on your own servers or hosted by the vendor, and existing OpenAI SDK code only needs a base URL and key change.

Source: Hacker News AI · layerx.xyz