SageMaker HyperPod adds three-tier data capture
Understand this story
AWS releases SageMaker HyperPod Data Capture, which can record online inference inputs and outputs to S3 at three independent levels: endpoint, load balancer, and model Pod. This capability is currently available for HyperPod clusters with the inference operator enabled; to use it, add the dataCapture configuration to an InferenceEndpointConfig or JumpStartModel resource and deploy. To complete a fine-tuning loop on the same cluster, the amazon-sagemaker-hyperpod-training-operator EKS add-on must also be installed. AWS says training nodes require outbound internet access (for example, through a NAT gateway), and pulling the vllm/vllm-openai image from Docker Hub also requires outbound access.
AI overview based on source reports · Updated 19 hr ago
Coverage timeline
Follow the coverage to explore different perspectives.
- AWS Machine Learning BlogBuild a model improvement flywheel with Amazon SageMaker HyperPod Data Capture
Amazon SageMaker HyperPod now offers Data Capture, which records live inference inputs and outputs to Amazon S3 at three independent tiers (SageMaker endpoint, load balancer, and model pod) with minimal latency impact and no impact on model availability. The post walks through a single-cluster example in which a home-appliance support router's production traffic is captured at the model-pod tier, analyzed with Amazon Athena to surface a new-product blind spot, labeled with a SQL policy, and used to fine-tune the router on a second GPU. Accuracy is measured before and after on data that could not have existed before deployment, with scripts, manifests, SQL, and synthetic datasets in an accompanying GitHub repository.
Activity for this story
- Current comparable activity
- 6
- Peak comparable activity
- 10Nov 14 03:00
- Change in the past 24 hours
- –
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.