49B low precision active parameters
104B total active parameters
47% of active parameters are low precision
Output1.17B params (bf16)
+residual add (no params)
S66M params (bf16)
133M params (MXFP4)
233M params (MXFP4)
333M params (MXFP4)
433M params (MXFP4)
533M params (MXFP4)
633M params (MXFP4)
733M params (MXFP4)
routed experts: top-16 of 896
2 shared experts
Router6.4M params (bf16)
AttentionKDA ×3  /  Gated MLA ×1KDA 444M or MLA 232M (bf16)
Embedding1.17B params (bf16), 1 active row = 7,168
1 of 93 layers
MXFP4 (low precision) bf16