Initializing Techhub.cafe

juniorAWS Machine Learning

What role does Amazon S3 play in an AWS machine learning workflow?

Updated Aug 1, 2026

Short answer

S3 is the durable backbone that connects every stage: raw data, engineered features, training data, model artefacts, batch predictions and monitoring captures all live there, letting compute stay ephemeral.

Deep explanation

Because SageMaker compute is created and destroyed per job, S3 is the only thing that persists across stages. Understanding the data flow is understanding the platform:

  1. Raw data lands in S3 (often via Kinesis Firehose, DMS or direct upload).
  2. A Processing job reads raw data, writes engineered features back to S3 (commonly Parquet).
  3. A training job reads features, writes model.tar.gz to S3.
  4. Deployment reads that artefact; batch transform writes predictions back to S3.
  5. Data capture on the endpoint writes live requests/responses to S3 for Model Monitor.

Input modes are the detail that separates a junior from a mid-level answer:

  • File mode (default) — the whole dataset is copied to the instance's EBS volume before training starts. Simple; slow startup for large data; limited by disk size.
  • FastFile mode — files are streamed on demand with a POSIX-like interface. Training starts almost immediately; ideal for large datasets read sequentially.
  • Pipe mode — data is streamed as a Unix pipe. Older, being superseded by FastFile for most cases.
Python
from sagemaker.inputs import TrainingInput


# 400 GB of images: File mode would spend ~30 minutes just downloading
train_input = TrainingInput(
    s3_data="s3://my-lake/features/train/",
    input_mode="FastFile",           # stream instead of pre-download
    s3_data_distribution_type="ShardedByS3Key",  # split across instances
)


estimator.fit({"train": train_input})

ShardedByS3Key is important for distributed training: with the default FullyReplicated, every instance downloads the entire dataset, so a 4-node cluster reads 4× the data and each node trains on everything. Sharding splits objects across nodes so each sees a distinct slice.

Organise buckets so lifecycle and access control are easy: s3://company-ml/{project}/{stage}/{version}/. Use Parquet with partitioning for anything queried by Athena.

Real-world example

A media company keeps ten years of clickstream in S3. Glue crawls it, Athena queries partitions to build a training slice, a Processing job converts it to Parquet, and training runs in FastFile mode over 2 TB — starting in seconds instead of waiting an hour for a download. S3 lifecycle rules move raw logs older than 90 days to Glacier Instant Retrieval, cutting storage cost substantially while keeping features in Standard.

Common mistakes

  • - Using File mode for very large datasets, so instances sit idle (and billing) during a long download.
  • - Leaving `s3_data_distribution_type` at the default in distributed training, so every node trains on the full dataset and you get no speedup.
  • - Storing training data as many small files — thousands of tiny objects destroy throughput. Aim for files in the 64–512 MB range.
  • - Putting the S3 bucket in a different region from training, incurring cross-region transfer cost and latency.
  • - Using CSV where Parquet would cut both storage and scan cost dramatically.

Follow-up questions

  • How do you give a training job access to an S3 bucket?
  • Why prefer Parquet over CSV for training data?
  • What is the difference between FullyReplicated and ShardedByS3Key?

More AWS Machine Learning interview questions

View all →