Alibaba releases multimodal Qwen3.5 mixture of experts model – News

Alibaba Group Holding Ltd. today released an artificial intelligence model that it says can outperform GPT-5.2 and Claude 4.5 Opus at some tasks.

The new algorithm, Qwen3.5, is available on Hugging Face under an open-source license.

By default, Qwen3.5 is capable of processing prompts with up to 262,144 tokens. Developers can nearly quadruple that number by applying customizations. Prompts may include text in more than 210 languages and dialects along with images such as data visualizations.

Qwen3.5 is a mixture of experts model, which means that comprises multiple neural networks that are optimized for different tasks. When the LLM receives a prompt, it uses 10 of its neural networks to generate an answer. Activating only some components of a model to process prompts is more hardware-efficient than running input through all its artifial neurons. Qwen3.5 has a total of 397 billion parameters, 17 billion of which are used per prompt.

Alibaba has also equipped the model with several other optimizations designed to boost its efficiency.

A large language model’s attention heads, the mechanisms that it uses to determine which data points to take into account when making a decision, usually scale quadratically. That means doubling the amount of data in a prompt quadruples the amount of RAM needed to produce a response. Qwen3.5 combines standard quadratic attention heads with so-called linear attention heads, which require considerably less memory.

The model also uses another efficiency-boosting technology called a gated delta network. The technology combines two deep learning techniques known as gating and the delta rule.

Gating enables an LLM to remove data that it doesn’t need for a task from its memory, which lowers hardware usage. The delta rule, in turn, is a version of the back-propagation algorithm that LLMs use to learn new tasks during training. It streamlines how the model updates its parameters during the learning process. Last year, Nvidia Corp. researchers determined that combining the two methods reduces the amount of hardware needed to train LLMs.

Alibaba compared Qwen3.5 to GPT-5.2 and Claude 4.5 Opus across more than 30 benchmarks. The model outperformed both on IFBench, a test that measures how well LLMs follow user instructions. In other cases, Qwen3.5 bested one of the LLMs but not the other. For example, it topped the score that Claude 4.5 Opus set on the HMMT reasoning benchmark but fell behind GPT-5.2.

Alibaba says that Qwen3.5 is also adept at processing multimodal data. It outperformed Qwen3-VL, a model built specifically for image analysis tasks, across several visual reasoning and coding benchmarks.

Image: Alibaba