How do you optimise data loading for training on very large datasets?

Match the input mode to the access pattern (FastFile for large sequential reads, FSx for Lustre for repeated random access), size files in the hundreds of megabytes, shard across instances, and overlap loading with compute so GPUs are never waiting.

On large training runs the GPU is frequently idle waiting for data, and teams respond by buying more GPUs — which makes utilisation worse. Diagnose before scaling.

Input modes and when each wins