What is the difference between a SageMaker training job and an inference endpoint?
Updated Aug 1, 2026
Short answer
A training job is a short-lived batch cluster that produces a model artefact in S3 and then terminates; an endpoint is a long-running HTTPS service that loads that artefact and serves predictions until you delete it.
Deep explanation
This is a billing and lifecycle question disguised as a definitions question, and it is where most junior AWS bills go wrong.
Training job — you submit it, AWS provisions instances, runs your container to completion, uploads model.tar.gz to S3, and destroys the instances. Billing is per-second for the duration of the job. If it crashes, it stops billing. You cannot "connect" to a training job; it is a batch workload.
Endpoint — you create it, AWS provisions instances and keeps them running, loading your model into memory. It exposes an HTTPS URL you call via InvokeEndpoint. Billing continues every second the endpoint exists, whether or not a single request arrives. It stops only when you call delete_endpoint.
The three objects behind an endpoint trip people up:
- Model — the artefact S3 URI plus the inference container image.
- EndpointConfig — instance type, count, and one or more production variants with traffic weights.
- Endpoint — the running deployment of a config.
That separation is what makes safe deployments possible: you create a new config and call update_endpoint, and traffic shifts without downtime.
# Training: ephemeral, pay for ~10 minutesestimator.fit({"train": train_s3_uri})print(estimator.model_data) # s3://.../model.tar.gz
# Deployment: persistent, pay per hour until deletedpredictor = estimator.deploy( initial_instance_count=1, instance_type="ml.m5.large", endpoint_name="fraud-scorer-v1",)
result = predictor.predict({"amount": 240.0, "merchant_risk": 0.7})
# The line juniors forget — this is what stops the meterpredictor.delete_endpoint()Logic: fit() is synchronous but the cost ends when it returns. deploy() returns quickly, but the cost starts when it returns and never ends on its own.
Real-world example
A team prototypes a churn model, deploys an ml.m5.xlarge endpoint to demo it, and moves on. Training cost them about $0.50. The forgotten endpoint costs roughly $170/month indefinitely. The standard fixes are a scheduled Lambda that deletes endpoints tagged env=dev outside working hours, or using serverless inference for demos so idle time costs nothing.
Common mistakes
- - Forgetting `delete_endpoint()` — by far the most common AWS ML cost leak.
- - Assuming an endpoint stops charging when traffic stops. It does not
- only serverless inference scales to zero.
- - Thinking you must retrain to change instance type. You create a new EndpointConfig from the same Model and update the endpoint.
- - Deploying straight from `estimator.deploy()` in production, which couples deployment to the training session rather than going through the Model Registry.
Follow-up questions
- How would you serve predictions without paying for idle capacity?
- Can two models share one endpoint?
- What does `model.tar.gz` actually contain?