seniorGradient Descent
What is bias-variance tradeoff in Gradient Descent optimization?
Updated May 16, 2026
Short answer
Bias-variance tradeoff describes error decomposition in model learning during optimization.
Deep explanation
Gradient Descent indirectly controls bias and variance through model complexity, regularization, and training dynamics. Early stopping increases bias but reduces variance, while prolonged training reduces bias but may increase variance due to overfitting.
Real-world example
Choosing training duration in neural network models.
Common mistakes
- Assuming longer training always improves performance.
Follow-up questions
- What is high bias?
- What is high variance?