What is the role of linear algebra in neural networks?

Neural networks are compositions of matrix multiplications and nonlinear functions.

Each layer performs Wx + b operations, which are linear transformations followed by nonlinear activation. Deep learning is essentially repeated linear algebra transformations in high-dimensional space.