What is SM utilization and how does PyTorch affect it?

SM utilization measures how effectively GPU streaming multiprocessors are used during computation.

Low SM utilization occurs when kernels are too small, memory-bound, or poorly scheduled. PyTorch performance tools like torch.compile, batching, and fusion increase utilization by reducing kernel launch overhead and improving arithmetic intensity.