juniorApache Spark
What is Spark Deployment Mode (Client vs Cluster)?
Updated May 5, 2026
Short answer
Client mode runs the driver on the machine where the job is submitted; Cluster mode runs the driver inside the cluster.
Deep explanation
Client mode is for interactive shells/debugging. Cluster mode is for production, where the driver lives on a worker node managed by YARN/K8s to prevent network bottlenecking on the local machine.
Real-world example
Using Client mode in a dev laptop to test; using Cluster mode in an Airflow production pipeline.
Common mistakes
- Using Client mode in production, causing failure if the gateway node disconnects.
Follow-up questions
- Where do logs go in Cluster mode?