How does AWS SageMaker handle distributed training?

SageMaker uses distributed training by splitting data and model computation across multiple instances.

SageMaker supports data parallelism and model parallelism. In data parallelism, each worker trains on a subset of data. In model parallelism, large models are split across GPUs. It uses frameworks like Horovod and DeepSpeed for synchronization.