Knowledge Distillation

A process in which a smaller AI model (student) is trained to replicate the predictions of a larger, more complex model (professor), reducing size and computing needs while maintaining performance.

‍