Why do TensorFlow models behave differently during training vs inference?

Training includes dropout and batch normalization updates, while inference disables them.

During training, layers like dropout randomly deactivate neurons and batch normalization updates running statistics. During inference, dropout is disabled and batch norm uses learned statistics. If not handled correctly, model behavior becomes inconsistent.