Route coding agent requests to the right model on Amazon Bedrock with NVIDIA NeMo Switchyard

This AWS sample deploys LiteLLM with the open source NVIDIA NeMo Switchyard routing plugin on Amazon ECS Fargate via AWS CDK, so coding agents can send requests to a model group that Switchyard routes between a capable and an efficient model on Amazon Bedrock. It covers Stage routing (which reads recent tool activity and results to pick between two candidates) and Random routing (weighted traffic splits for A/B tests), plus a benchmark runner that measures cost per solved task against single-model baselines. The post notes the sample uses switchyard-server 0.3.0 and LiteLLM v1.102.0, is for non-production use, and that its configuration and APIs may change between releases.