QAD reduces KL against the original BF16 model
NVFP4 · attention + MLP · Qwen 3.8 27B · Internal multi-turn reasoning dataset
Rollout engine · held-out
Rollout engine · training
Training actor · held-out
No QAD · rollout held-out
QAD reduces KL against the original BF16 model
0.00
0.01
0.02
0.03
0.04
No QAD
NVFP4
0.03485
0.03446
0.03605
Normal QAD
Top-500 KL
0.02064 −40.8%
0.02006
0.02181
Deployment-informed
Top-500 KL
0.02235 −35.9%
0.02186
0.02393
Normal QAD
One-hot CE
0.02576 −26.1%
0.02567
0.02717
Deployment-informed
One-hot CE
0.03128 −10.3%
0.03137
0.03285
Full-vocabulary forward KL (nats) · lower is better