How does TensorFlow handle memory fragmentation in GPU training workloads?

TensorFlow uses memory pooling and allocator strategies to reduce GPU fragmentation.

GPU memory fragmentation occurs when allocations and deallocations leave unusable gaps. TensorFlow uses a caching allocator to reuse memory blocks and reduce fragmentation. However, long-running training jobs with dynamic tensor shapes can still suffer fragmentation and OOM errors.