AWS publishes tutorial on Run:ai GPU memory fractioning on EKS
Understand this story
AWS has published a hands-on tutorial on sharing GPU memory on Amazon EKS through NVIDIA Run:ai GPU Fractioning. The approach uses the GPU Fractioning Operator to enable software-level sharing based on MPS, setting a mandatory memory limit for each Pod and providing independent metrics. The tutorial requires GPU capacity to be provisioned in advance, and custom images cannot run on nodes managed by Amazon EKS Auto Mode because Auto Mode selects and manages AMIs itself. The tutorial validated only G5/A10G configurations. Following the tutorial creates billable resources, including the EKS control plane, NAT gateway, instances, volumes, and AMI snapshots.
AI overview based on source reports · Updated 18 hr ago
Coverage timeline
Follow the coverage to explore different perspectives.
- AWS Machine Learning BlogUsing NVIDIA Run:ai GPU Fractioning on Amazon EKS
This AWS post walks through configuring NVIDIA Run:ai GPU Fractioning on Amazon EKS using a custom Amazon Linux 2023 image, with a validated example of one g5.xlarge instance and an NVIDIA A10G GPU. It establishes a whole-GPU baseline, runs two Pods with 8 GiB memory contracts each allocating 6 GiB on the same GPU, inspects per-Pod metrics, and verifies rejection of a 9 GiB allocation. The walkthrough uses KAI Scheduler and GPU Fractioning Operator 0.1.7 with NVIDIA driver 615.71.10, and notes that fractioning adds enforced per-Pod memory limits but does not provide fault isolation or proportional compute.
Activity for this story
- Current comparable activity
- 6
- Peak comparable activity
- 10Nov 20 01:00
- Change in the past 24 hours
- –
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.