Interesting study on model distillation.

A distilled student model copies how its teacher reasons rather than what it knows, so teacher lineage matters far more than teacher benchmark scores.

A weaker teacher model built from your student's own base model will take it further than a stronger teacher from a different family.

This paper finds, the data barely matters. Train on problems the teacher always solves, never solves, or a random mix, and you land in the same place. Even grade-school math gets you over 80% of the benefit.

A teacher can teach a problem it can't solve itself, because what's moving across is a habit of reasoning.