How do you optimise a batch scoring job over billions of records?

Parallelise across many instances with sharded input, tune payload batching and concurrency per instance, use Spot for the fleet, right-size the instance to the model, and partition output so downstream consumers can read it efficiently.

Batch scoring is throughput-bound, and the levers differ from real-time inference — latency per record is irrelevant, records per dollar is everything.

Batch transform tuning is mostly three parameters, and defaults leave a lot on the table:

```python transformer = model.transformer( instance_count=40, # near-linear scaling instance_type="ml.c5.4xlarge", strategy="MultiRecord", # many records per request, not one max_payload=6, # MB per request max_concurrent_transforms=8, # parallel requests per instance assemb…