Dictionary What's New
Model distillation
What it means
Compressing a big model's knowledge into a smaller, faster, cheaper one - the small model learns by imitating the big one's answers.
An everyday example
A master chef writing a simplified cookbook: the home cook won't match every dish, but gets 90% of the flavor at 10% of the effort.