What is model parallelism in neural networks?

Model parallelism splits a single model across multiple devices.

Different layers or parts of a model are placed on different GPUs to handle very large models.