SKD

Notes on Speculative Knowledge Distillation - a distillation method that lets the student draft tokens on-policy and only resamples from the teacher the ones that fall outside its top-K, braiding realistic states with reliable feedback at the token level.