How do you deploy a trained model to a real-time SageMaker endpoint?

Updated Aug 1, 2026

Short answer

Create a Model (artefact plus inference image), an EndpointConfig (instance type, count, variants), then an Endpoint. Clients call InvokeEndpoint over HTTPS and you pay per instance-hour until the endpoint is deleted.

Deep explanation

The three-object structure is not bureaucracy — it is what makes zero-downtime updates possible. Because the Model and the EndpointConfig are separate from the Endpoint, you can build a new configuration and swap traffic to it atomically.

Python

1from sagemaker.model import Model
2from sagemaker.serializers import JSONSerializer
3from sagemaker.deserializers import JSONDeserializer
4
5model = Model(
6    image_uri=inference_image,                 # container that serves the model
7    model_data="s3://my-bucket/models/fraud/model.tar.gz",
8    role=role,
9    name="fraud-model-v3",
10)
11
12predictor = model.deploy(
13    initial_instance_count=2,                  # 2+ for availability across AZs
14    instance_type="ml.m5.xlarge",
15    endpoint_name="fraud-scorer",
16    serializer=JSONSerializer(),
17    deserializer=JSONDeserializer(),
18    data_capture_config=DataCaptureConfig(     # needed later by Model Monitor
19        enable_capture=True,
20        sampling_percentage=20,
21        destination_s3_uri="s3://my-bucket/capture/fraud",
22    ),
23)

For a custom container, the inference script implements four hooks. Understanding their order is the whole request lifecycle:

Python

1# inference.py
2import joblib, os, json
3
4def model_fn(model_dir):                 # once, at container start
5    return joblib.load(os.path.join(model_dir, "model.joblib"))
6
7def input_fn(request_body, content_type):     # per request: parse
8    if content_type == "application/json":
9        return json.loads(request_body)["features"]
10    raise ValueError(f"Unsupported content type: {content_type}")
11
12def predict_fn(data, model):                  # per request: infer
13    return model.predict_proba([data])[0][1]
14
15def output_fn(prediction, accept):            # per request: serialise
16    return json.dumps({"fraud_probability": float(prediction)}), accept

model_fn runs once per container, so load heavy objects there, never inside predict_fn. Calling it per request is the single most common cause of an endpoint that "works" but has terrible latency.

Invoking from an application:

Python

1import boto3, json
2runtime = boto3.client("sagemaker-runtime")
3
4resp = runtime.invoke_endpoint(
5    EndpointName="fraud-scorer",
6    ContentType="application/json",
7    Body=json.dumps({"features": [240.0, 0.7, 3, 1]}),
8)
9print(json.loads(resp["Body"].read()))

Updating safely: create a new EndpointConfig and call update_endpoint. SageMaker provisions the new fleet, shifts traffic, then removes the old one — no downtime and automatic rollback if the new variant fails health checks.

Real-world example

A payments company serves a fraud model at p99 under 100 ms with two ml.c5.xlarge instances across availability zones, fronted by API Gateway and Lambda for authentication and request shaping. Deployments go through a new EndpointConfig with a canary variant at 10% traffic for fifteen minutes; a CloudWatch alarm on ModelLatency and Invocation5XXErrors triggers automatic rollback before customers notice.

Common mistakes

- Loading the model inside `predict_fn` instead of `model_fn`, adding seconds to every request.
- Deploying a single instance in production — no redundancy, and any AZ event takes the service down.
- Deleting and recreating an endpoint to update it, causing an outage where `update_endpoint` would have been seamless.
- Ignoring the 60-second `InvokeEndpoint` limit and the payload size limit
long or large inference belongs on asynchronous inference.
- Enabling data capture only after a drift incident, when the baseline you needed was never recorded.

Follow-up questions

How do you update a model with no downtime?
What do you do when inference takes longer than 60 seconds?
How would you serve hundreds of per-customer models cost-effectively?

Short answer

Deep explanation

Real-world example

Common mistakes

Follow-up questions

More AWS Machine Learning interview questions