ToMoE: converting dense LLMs to Mixture-of-Experts via dynamic structural pruning

ToMoE is a research paper proposing a method to convert existing dense large language models into Mixture-of-Experts architectures through dynamic structural pruning, without full retraining from scratch. The approach routes inputs to identified parameter subsets that behave as specialized experts. If reproduced broadly, the technique could lower the cost of obtaining MoE efficiency gains from already-trained dense models.