Why do deep neural networks suffer from vanishing gradients in linear algebra terms?

Repeated multiplication of small Jacobians shrinks gradient magnitude.

Backpropagation involves repeated multiplication of Jacobian matrices. If eigenvalues of these matrices are < 1, gradients exponentially decay across layers, leading to vanishing gradients.