Distillation vs Quantization
Distillation (Model Distillation) and Quantization (Model Quantization) both come up in ai & ml conversations and get confused. Here's the plain-English difference, side by side, so you can use each one with confidence.
The key difference: Distillation refers to model distillation, while Quantization refers to model quantization — they describe different things even when they show up in the same sentence.
Distillation — Model Distillation
Training a smaller "student" model to mimic a larger "teacher" model. Distillation is how you take a flagship-quality experience and ship it at a price you can defend in a P&L.
Full Distillation definition →
Quantization — Model Quantization
Reducing the numerical precision of model weights (for example from 16-bit to 4-bit) to shrink size and speed up inference. The cheapest unlock for running serious models on modest hardware.
Full Quantization definition →
When to use Distillation
Reach for "Distillation" when the conversation is specifically about model distillation. Training a smaller "student" model to mimic a larger "teacher" model. Distillation is how you take a flagship-quality experience and ship it at a price you can defend in a P&L.
When to use Quantization
Reach for "Quantization" when the conversation is specifically about model quantization. Reducing the numerical precision of model weights (for example from 16-bit to 4-bit) to shrink size and speed up inference. The cheapest unlock for running serious models on modest hardware.
FAQs
What is the difference between Distillation and Quantization?
Distillation stands for Model Distillation — Training a smaller "student" model to mimic a larger "teacher" model. Distillation is how you take a flagship-quality experience and ship it at a price you can defend in a P&L. Quantization stands for Model Quantization — Reducing the numerical precision of model weights (for example from 16-bit to 4-bit) to shrink size and speed up inference. The cheapest unlock for running serious models on modest hardware.
Are Distillation and Quantization the same thing?
No. They're often used in the same conversation because they're related, but they describe different concepts. Distillation = Model Distillation. Quantization = Model Quantization.
When should I use Distillation vs Quantization?
Use Distillation when you're specifically referring to model distillation. Use Quantization when the topic is model quantization.