The Stack
AI 101 — What is knowledge distillation?
Knowledge distillation is a way of training a small "student" AI model to imitate a big "teacher" model — so the student picks up much of the teacher's skill while being far smaller, faster, and cheaper to run. If you have read tech news lately,