Can I Have Your Order? Monte-Carlo Tree Search for Slot Filling Ordering in Diffusion Language Models

Researchers have introduced McDiffuSE, a novel framework that applies Monte Carlo Tree Search to optimize slot infilling order in Masked Diffusion Models, addressing performance variability issues in mathematical and code reasoning tasks, as detailed in their new paper released on arXiv (2602.12586v1). The paper highlights that while plan-and-infill decoding in Masked Diffusion Models shows promise for complex reasoning tasks, the performance is highly sensitive to the order in which slots are filled, leading to inconsistent outputs. McDiffuSE tackles this by formulating slot selection as a decision-making problem, using look-ahead simulations to evaluate partial completions and determine optimal infilling sequences. This approach allows the model to explore different completion paths and select the most promising one, reducing output variance and improving reliability. The researchers demonstrate that McDiffuSE can significantly enhance the consistency and quality of outputs from diffusion language models when handling mathematical and code generation tasks, representing an important advancement toward making these models more practical for technical applications where precision matters.