Z.ai and Alibaba Release Near-Identical MoE Models Within a Day
This digest was compiled by AI from multiple sources — links to the originals are below.
Z.ai released GLM-5.3-Flash, a 320B-parameter multimodal MoE model, and Alibaba released Qwen3.8-Flash-Next, a 125B model previewing Qwen4, within a day of each other. Both models independently converged on a 3:1 hybrid of linear and full attention, a compressed context indexer capped at 2048 tokens, and the Muon optimizer. GLM-5.3-Flash lists at $0.15 per million input tokens, while Qwen3.8-Flash-Next required about one-ninth the compute of Qwen3.7-Plus.
Key Facts
- GLM-5.3-Flash is a 320B-parameter multimodal MoE model with 18B active parameters, released under the MIT license on Hugging Face.
- Qwen3.8-Flash-Next is a 125B model with 6B active parameters that previews the Qwen4 architecture and includes an additional 51B n-gram embedding table.
- Both models use a 3:1 hybrid of linear and full attention, a compressed context indexer capped at 2048 tokens, and the Muon optimizer.
- GLM-5.3-Flash was trained on a 30T-token multimodal corpus and serves a 1M-token context window.
- Qwen3.8-Flash-Next has a native context of 262,144 tokens, extensible to 1M with YaRN, and required about one-ninth the compute of Qwen3.7-Plus.
The Two Releases
GLM-5.3-Flash is the first natively multimodal model in the GLM-5 series, released under the MIT license on Hugging Face. Z.ai tested it anonymously as Ox Alpha on OpenRouter, where it became the most popular model of the week. Z.ai says it outperforms GLM-5.2 across benchmarks at one-tenth the price, while approaching Claude Opus 4.8 on coding and agentic benchmarks. List pricing is $0.15 per million input tokens and $0.50 per million output tokens. Qwen3.8-Flash-Next plays the role Qwen3-Next played for Qwen3.5: an early public preview of the next architecture family.
Convergent Architecture Choices
GLM-5.3-Flash stacks 45 layers: 34 linear-attention layers and 11 full-attention layers, per the shipped config. Qwen3.8-Flash-Next stacks 48 layers in a repeating block of 3 Gated DeltaNet layers plus 1 Qwen Sparse Attention layer, per the vLLM recipe. Both land on the same 3:1 ratio of linear to full attention layers. The linear layers compress all history into a fixed-size recurrent state, keeping compute per token constant regardless of context length. GLM uses Kimi Delta Attention (KDA), introduced by Moonshot AI’s Kimi Linear, while Qwen uses its own Gated DeltaNet (GDN) with per-head gating.
1 source
Z.ai and Alibaba Release Near-Identical MoE Models Within a Day
