Quantization — Model Quantization
Definition: Reducing the numerical precision of model weights (for example from 16-bit to 4-bit) to shrink size and speed up inference. The cheapest unlock for running serious models on modest hardware.
Example
4-bit quantization let the 70B model run on a single workstation GPU — same answers, no cloud bill.
When you'll hear it
Quantization shows up most often in AI strategy reviews, model evaluation discussions, and product roadmap meetings. When someone uses it, they're usually referring to model quantization — and they expect the room to already know what that means.
FAQs
What does Quantization stand for?
Quantization stands for Model Quantization.
What does Quantization mean in AI and machine-learning?
Reducing the numerical precision of model weights (for example from 16-bit to 4-bit) to shrink size and speed up inference. The cheapest unlock for running serious models on modest hardware.
Where will I hear Quantization used at work?
Quantization comes up most often in AI strategy reviews, model evaluation discussions, and product roadmap meetings. It's used as shorthand for model quantization, so people assume you already know the term.