How does data partitioning strategy in distributed ML affect bias and variance?

Poor data partitioning increases bias due to skewed subsets and increases variance due to inconsistent local model updates.

In distributed ML systems, data partitioning determines how training data is split across nodes. If partitions are not representative of the global distribution, local models learn biased patterns, increasing global bias after aggregation.

Non-IID (non-independent and identically distributed) partitions are especially problematic in federated learning, where each client has unique data distributions. This leads to high variance in gradients and unstable convergence.…