How do you fine-tune a large language model on SageMaker cost-effectively?

Use parameter-efficient fine-tuning such as LoRA or QLoRA so you train a small fraction of the weights, combine with bf16, FSDP sharding, gradient checkpointing and Spot instances, and evaluate against a held-out set before deciding fine-tuning was needed at all.

The senior instinct is to challenge the premise first: most tasks that people fine-tune for are better solved by prompting or RAG. Fine-tuning teaches format, tone and task behaviour; it is a poor and expensive way to teach facts, which change and are better retrieved. Establish a prompted baseline before spending on training.

When fine-tuning is genuinely warranted, full fine-tuning of a 7B+ model is rarely necessary.

LoRA freezes the base weights and learns low-rank adapters ΔW = BA where B ∈ R^(d×r), A ∈ R^(r×k) with r typically 8–64.…