DBRX MoE-A
LLMLLM
Smallest DBRX family member: 7.7B total parameters, 2.2B active. Used internally by Databricks to study MoE training efficiency. Achieves 30.5% on the Databricks Gauntlet with 3.7x fewer FLOPs than MPT-7B.
Technical specification
Context window
Parameters
License
Tools
Fine-tuning
Weights access
Hardware requirementsInternal Databricks research model; no public checkpoint available.
Last updated: May 4, 2026
Modalities
Input
Text
Output
Text
Code