Design a near-real-time streaming inference pipeline on AWS

Updated Aug 1, 2026

Short answer

Ingest with Kinesis Data Streams, enrich and window with Managed Service for Apache Flink, call a SageMaker endpoint for scoring with batching and retries, and write results to a downstream stream or store — sizing shards and concurrency so the endpoint is never the bottleneck.

Deep explanation

Streaming inference fails in specific, predictable ways, and the architecture is mostly about avoiding them.

TypeScript

1Producers → Kinesis Data Streams → Managed Flink (enrich + window)
2                                        │
3                                        ├→ SageMaker endpoint (scored in micro-batches)
4                                        │
5                                        ├→ Kinesis (scored events) → consumers
6                                        └→ Firehose → S3 (audit + training data)