How does PyTorch handle memory reuse in the caching allocator?

PyTorch reuses freed GPU memory blocks using a caching allocator to reduce allocation overhead.

Instead of returning memory to CUDA immediately, PyTorch keeps freed blocks in a cache. Future allocations reuse these blocks if sizes match, reducing fragmentation and allocation latency.