Apple open sourced DiffuCoder, a diffusion large language model (dLLM) fine-tuned for coding tasks. DiffuCoder is based on Qwen-2.5-Coder and outperforms other code-specific LLMs on several coding benchmarks.
Unlike typical LLMs, which generate text auto-regressively and “left-to-right,” dLLMs generate text by de-noising an entire sequence in parallel, which can mean faster generation. Apple’s researchers developed DiffuCoder so they could investigate the best strategies for dLLM fine-tuning and inference. In their research, they developed a variation of the Group Relative Policy Optimization (GRPO) fine-tuning technique that they call coupled-GRPO which improves the model’s performance. On the MBPP coding benchmark, DiffuCoder outperformed Gemini Diffusion and was “competitive” with GPT-4o. According to Apple,
By using a novel coupled-sampling strategy, our method provides a more accurate likelihood estimation. Coupled-GRPO significantly boosts DiffuCoder’s performance, demonstrating the effectiveness of RL methods aligned with diffusion principles. Our work provides the community with a deeper understanding of dLLMs and lays a strong foundation for future explorations of dLLMs in complex reasoning and generation tasks.
Most LLMs, like OpenAI’s GPT models, generate text auto-regressively by predicting a single next token to append to a sequence, then feeding the new sequence back as input. dLLMs take an approach similar to image-generation models like DALL-E: they start with a noisy sequence and iteratively de-noise it. This allows dLLMs to generate output much faster than autoregressive LLMs: up to five times faster in the case of Gemini Diffusion. Furthermore, they are not constrained to produce text left-to-right. Instead, they can perform a “global planning of content,” which can be an advantage in coding tasks.
One key outcome of Apple’s research was creating autoregressive-ness (AR-ness) metrics, which measures how closely a model follows the left-to-right pattern of LLMs. They found that dLLMs often do exhibit a high degree of AR-ness, likely due to the inherent nature of text generation. However, when generating code, this metric drops.
They also found that increasing the sampling temperature affected the model’s AR-ness, by making the model more flexible in both its choice of tokens and token order. This improved its “pass@k” score on coding benchmarks. The researchers point to past work that shows that a RL fine-tuned model’s reasoning ability is “bounded by the base model’s pass@k sampling capabilities,” which suggested that DiffuCoder had “substantial” potential for improvement. This led to their development of coupled-GRPO RL training, which did improve DiffuCoder’s benchmark results, by over six percentage points in some cases.
In a discussion on Hacker News, one user wrote:
A diffusion model comes with a lot of benefits in terms of parallelization and therefore speed; to my mind the architecture is a better fit for coding than strict left to right generation…Overall, interesting. At some point these local models will get good enough for “real work” and they will be slotted in at API providers rapidly. Apple’s game is on-device; I think we’ll see descendants of these start shipping with Xcode in the next year as just part of the coding experience.
The DiffuCoder code is available on GitHub. The model files can be downloaded from Huggingface.
