How does dropout act as a variance reduction technique in neural networks?

Dropout reduces variance by randomly disabling neurons during training, preventing co-adaptation and overfitting.

Dropout is a regularization technique where random subsets of neurons are deactivated during training. This forces the network to learn redundant representations rather than relying on specific neurons. As a result, the model becomes more robust and less sensitive to noise, reducing variance.

At inference time, all neurons are active with scaled weights, effectively averaging multiple implicit subnetworks. This acts like an ensemble of neural networks, improving generalization.