Qwen3-32B

Qwen3-32B is the largest dense large language model in the Qwen3 family developed by the Qwen team at Alibaba Cloud, released on 29 April 2025 under the Apache 2.0 license.

Architecture

The model has 32.8B parameters (31.2B non-embedding) and 64 transformer decoder layers. It uses Grouped Query Attention with 64 query (Q) heads and 8 key/value (KV) heads, SwiGLU, Rotary Position Embeddings (RoPE), RMSNorm with pre-normalization and QK-Norm. The QKV bias used in Qwen2 was removed. The model natively supports a 32,768-token context, extendable to 131,072 tokens via YaRN.

Thinking modes

A defining feature of Qwen3 is its hybrid operation: a thinking mode for complex reasoning, math and coding, and a non-thinking mode for fast dialogue. The modes can be switched dynamically based on the query or the chat template.

Training

The model was pretrained on roughly 36 trillion tokens covering 119 languages and dialects, followed by a multi-stage post-training pipeline (long chain-of-thought SFT and reinforcement learning). Qwen3-32B has strong agentic and external tool-integration capabilities. Weights are released under the Apache 2.0 license.