LoRA vs SFT
LoRA (Low-Rank Adaptation) and SFT (Supervised Fine-Tuning) both come up in ai & ml conversations and get confused. Here's the plain-English difference, side by side, so you can use each one with confidence.
The key difference: LoRA refers to low-rank adaptation, while SFT refers to supervised fine-tuning — they describe different things even when they show up in the same sentence.
LoRA — Low-Rank Adaptation
A fine-tuning technique that adapts a large model by training small adapter layers instead of the full network. Cheap, fast, and swappable — the workhorse of practical model customization.
SFT — Supervised Fine-Tuning
Training a base model on curated input-output pairs to specialize its behavior. SFT is where most domain models actually get their personality — long before RLHF cleans up the rough edges.
When to use LoRA
Reach for "LoRA" when the conversation is specifically about low-rank adaptation. A fine-tuning technique that adapts a large model by training small adapter layers instead of the full network. Cheap, fast, and swappable — the workhorse of practical model customization.
When to use SFT
Reach for "SFT" when the conversation is specifically about supervised fine-tuning. Training a base model on curated input-output pairs to specialize its behavior. SFT is where most domain models actually get their personality — long before RLHF cleans up the rough edges.
FAQs
What is the difference between LoRA and SFT?
LoRA stands for Low-Rank Adaptation — A fine-tuning technique that adapts a large model by training small adapter layers instead of the full network. Cheap, fast, and swappable — the workhorse of practical model customization. SFT stands for Supervised Fine-Tuning — Training a base model on curated input-output pairs to specialize its behavior. SFT is where most domain models actually get their personality — long before RLHF cleans up the rough edges.
Are LoRA and SFT the same thing?
No. They're often used in the same conversation because they're related, but they describe different concepts. LoRA = Low-Rank Adaptation. SFT = Supervised Fine-Tuning.
When should I use LoRA vs SFT?
Use LoRA when you're specifically referring to low-rank adaptation. Use SFT when the topic is supervised fine-tuning.