AWS Machine Learning Blog· Dan Ferguson·· 2 days agoAI score42
Introducing the Amazon SageMaker JumpStart Model Leaderboard
Introducing the Amazon SageMaker JumpStart Model Leaderboard
AI summary
AWS launched the SageMaker JumpStart Model Leaderboard, a view in SageMaker Studio that compares 70+ open-weight models from providers including Meta, Google, Qwen, IBM, and Arcee on quality, speed, and cost. Quality scores come from three benchmarks (BIG-Bench Hard, GPQA Diamond, and MMLU-Pro) run by AWS with the same evaluation setup for every model, while performance and cost metrics are measured by sending a standardized test workload to endpoints using each JumpStart deployment configuration. Users can filter models, select a use case profile and optimization goal, and then deploy or customize a model.
Source: AWS Machine Learning Blog · aws.amazon.com