What is Gradient Descent and how does it optimize neural networks?

Gradient Descent is an optimization algorithm that iteratively updates model parameters in the direction of the negative gradient of the loss function to minimize error.

Gradient Descent is the fundamental optimization method used to train neural networks by minimizing a loss function.

Core idea:

Mathematical form: θ = θ - η ∇L(θ)

Where:

Why it works:

Types:

3.…