What usually stops PyTorch training from scaling out to a cluster is the container, not the model code. Every container needs the same CUDA runtime , a PyTorch build compiled against it, NCCL (NVIDIA Collective Communications Library), the Elastic Fabric Adapter (EFA) userspace libraries, the AWS OFI NCCL plugin that connects NCCL to EFA, and a Ray version the cluster operator agrees with.
What happened
Building that image and rebuilding it each time one of those pieces moves is a project of its own, and it has nothing to do with the model you set out to train. The Ray Train Deep Learning Container (DLC) is that image, built by AWS. AWS assembles the stack, tests it as one unit, publishes it in Amazon Elastic Container Registry (Amazon ECR) Public and keeps it patched.
Where the sources line up
Figure 1: One EKS cluster with a control plane managed by AWS, a CPU system node (m7i. xlarge) running the KubeRay operator and Ray head, and two EFA-enabled g6. 12xlarge nodes in one Availability Zone. KubeRay builds it from one RayCluster manifest, Ray Train places eight workers, one per GPU, and inter-node collectives cross over EFA.
Practical impact for readers
A single tagged image runs as both the Ray head and the Ray workers, whether KubeRay starts it on Amazon Elastic Kubernetes Service (Amazon EKS) or you start it yourself on Amazon Elastic Compute Cloud (Amazon EC2). What you add is configuration: which instances, how many GPUs each pod gets, which network transport NCCL uses, and which extra Python packages a job needs. All of it is set from outside the image.
Who should pay attention now
This post walks through a sample that uses the DLC exactly as published to fine-tune Qwen2. 5-1. 5B with PyTorch Fully Sharded Data Parallel (FSDP) across eight NVIDIA L4 GPUs on two g6. 12xlarge instances. The training code is one function, the cluster is one RayCluster manifest plus a file of variables, and no image is built at any point. Because the DLC already carries the EFA stack, collectives between the two instances run over the EFA as soon as the pods request it. The entire solution is in this sample repository , and it uses the same KubeRay-on-EKS foundation as our multi-node Ray Serve walkthrough .
What is still unclear
The Ray Train DLC is the PyTorch training image with Ray’s training libraries on top. The tag this sample pulls, public. ecr. aws/deep-learning-containers/ray:train-ml-cuda-v1. 1 , replaces a build that would otherwise look like this:.
Latest comments
0No comments yet. You can start the conversation.