What are key differences between training and inference computation graphs in transformers?

Training graphs include backpropagation paths; inference graphs only perform forward computation.

During training, activations are stored for gradient computation, and autograd builds a full computation graph. In inference, only forward pass is executed with optimizations like KV caching and fused kernels. This difference significantly impacts memory and latency.