A technique where a smaller “student” model learns to mimic the behavior of a larger, more complex “teacher” model. Knowledge distillation enables deployment of efficient models with reduced computational requirements while maintaining much of the original model’s performance.
