Initializing Techhub.cafe

juniorAWS Machine Learning

What is Amazon Bedrock and how does it differ from hosting a model on SageMaker?

Updated Aug 1, 2026

Short answer

Bedrock is a serverless API for foundation models from providers such as Anthropic, Meta and Amazon, billed per token with no infrastructure to manage; SageMaker hosting means you provision and pay for the instances running the model yourself.

Deep explanation

Bedrock removes the hardest parts of using large models: capacity, scaling and provider integration. One API surface fronts many models, so switching from one to another is largely a model-ID change rather than a re-architecture.

What it provides beyond raw inference:

  • Knowledge Bases — managed RAG: point at S3, it chunks, embeds, stores vectors (OpenSearch Serverless, Aurora pgvector and others) and handles retrieval.
  • Guardrails — configurable filters for harmful content, denied topics, PII redaction and contextual grounding checks, applied independently of the model.
  • Agents — models that call your APIs/Lambda functions to complete multi-step tasks.
  • Provisioned Throughput — reserved capacity for predictable high volume, at lower effective per-token cost than on-demand.
Python
import boto3, json


bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")


# The Converse API normalises message format across providers
response = bedrock.converse(
    modelId="anthropic.claude-sonnet-4-20250514-v1:0",
    messages=[{
        "role": "user",
        "content": [{"text": "Summarise this support ticket in one sentence: ..."}],
    }],
    inferenceConfig={"maxTokens": 300, "temperature": 0.2},
)
print(response["output"]["message"]["content"][0]["text"])
print(response["usage"])   # inputTokens / outputTokens — this is what you pay for

The economic distinction that matters in interviews: Bedrock scales to zero, SageMaker hosting does not. An idle Bedrock integration costs nothing; an idle ml.g5.12xlarge endpoint costs thousands per month. Conversely, at very high sustained volume a self-hosted smaller model can undercut per-token pricing — the crossover point is a real calculation, not a preference.

Use SageMaker hosting for foundation models when you need weights in your own VPC for compliance, a model Bedrock does not offer, or custom inference code such as an unusual quantisation or decoding strategy.

Real-world example

A SaaS company added a "summarise this thread" feature using Bedrock. Traffic is bursty — a few thousand calls at business hours, near zero overnight — so per-token pricing costs a fraction of a always-on GPU endpoint, and they shipped in days rather than standing up inference infrastructure. When one high-volume enterprise customer pushed sustained load, they moved that tenant to Provisioned Throughput for predictable cost and latency.

Common mistakes

  • - Assuming Bedrock trains on your prompts. It does not use your inputs or outputs to train the base models — but confirm the current terms rather than asserting it from memory in an interview.
  • - Ignoring token cost until the bill arrives. Long system prompts multiply across every request
  • measure `usage` from day one.
  • - Not requesting model access first — models must be explicitly enabled per account and region before calls succeed.
  • - Assuming every model is available in every region
  • availability varies and affects latency and data residency.
  • - Skipping Guardrails and relying only on prompt instructions for safety.

Follow-up questions

  • When does self-hosting a model on SageMaker beat Bedrock on cost?
  • What is Provisioned Throughput and when do you need it?
  • How do Bedrock Guardrails differ from prompting the model to behave?

More AWS Machine Learning interview questions

View all →