📝 Knowledge Distillation Layer That Lets the Student Decide 🔭

"Proposes a learnable knowledge distillation (letKD) layer for the student which improves KD with two distinct abilities: i) learning how to leverage the teacher's knowledge, enabling to discard nuisance information, and ii) feeding forward the transferred knowledge deeper." [gal30b+] 🤖 #CV

⚙️ https://github.com/adagorgun/letKD-framework
🔗 https://arxiv.org/abs/2309.02843v1 #arxiv

https://creative.ai/system/media_attachments/files/111/024/602/518/498/251/original/5853116792dde4fb.jpg

https://creative.ai/system/media_attachments/files/111/024/602/576/324/021/original/9571d8e6ee51e7ec.jpg