Hacker News AI· carlosmendes·· 46 min agoAI score58
AI Gateway: Manage your company's AI from one place
AI Gateway: Manage your company's AI from one place
AI summary
AI Gateway is a self-hosted, OpenAI-compatible endpoint that routes requests to your own GPU servers (vLLM, Ollama) first and falls back to providers like OpenAI, Groq and Together when needed. It adds per-user keys, token budgets and model access controls, plus a dashboard tracking tokens and cost per user, key, model and provider, and it does not log or store prompts and responses. Deployment is on your own servers or hosted by the vendor, and existing OpenAI SDK code only needs a base URL and key change.
Source: Hacker News AI · layerx.xyz