Distillation vs Inference
Distillation (Model Distillation) and Inference (Model Inference) both come up in ai & ml conversations and get confused. Here's the plain-English difference, side by side, so you can use each one with confidence.
The key difference: Distillation refers to model distillation, while Inference refers to model inference — they describe different things even when they show up in the same sentence.
Distillation — Model Distillation
Training a smaller "student" model to mimic a larger "teacher" model. Distillation is how you take a flagship-quality experience and ship it at a price you can defend in a P&L.
Full Distillation definition →
Inference — Model Inference
The act of running a trained model on new input to produce output. Inference cost — not training cost — is what actually shows up on the cloud bill once a feature is live.
When to use Distillation
Reach for "Distillation" when the conversation is specifically about model distillation. Training a smaller "student" model to mimic a larger "teacher" model. Distillation is how you take a flagship-quality experience and ship it at a price you can defend in a P&L.
When to use Inference
Reach for "Inference" when the conversation is specifically about model inference. The act of running a trained model on new input to produce output. Inference cost — not training cost — is what actually shows up on the cloud bill once a feature is live.
FAQs
What is the difference between Distillation and Inference?
Distillation stands for Model Distillation — Training a smaller "student" model to mimic a larger "teacher" model. Distillation is how you take a flagship-quality experience and ship it at a price you can defend in a P&L. Inference stands for Model Inference — The act of running a trained model on new input to produce output. Inference cost — not training cost — is what actually shows up on the cloud bill once a feature is live.
Are Distillation and Inference the same thing?
No. They're often used in the same conversation because they're related, but they describe different concepts. Distillation = Model Distillation. Inference = Model Inference.
When should I use Distillation vs Inference?
Use Distillation when you're specifically referring to model distillation. Use Inference when the topic is model inference.