Building a self-service LLM gateway with Amazon Bedrock and LiteLLM

This post describes a pattern for a self-service LLM gateway that runs LiteLLM on AWS Fargate to expose Amazon Bedrock models through an OpenAI-compatible API, using Bedrock Application Inference Profiles for per-model quota isolation and scoped IAM access. It covers configuration of developer tools, LiteLLM model mapping with per-token costs for budget enforcement, CDK constructs for inference profiles, Langfuse observability, CloudWatch TPM alarms, and lessons learned from scaling to several hundred developers.