A Better FP4 Gradient Quantizer That Training Couldn't Notice

Four-bit training is the current frontier of making LLM pretraining cheaper. NVIDIA’s Blackwell GPUs do 4-bit matrix math several times faster than 16-bit, but 4 bits means every number gets rounded to one of 16 values. The NVFP4 format’s menu is exactly {0, ±0.5, ±1, ±1.5, ±2, ±3, ±4, ±6}, stretched by one scale factor per block of 16 numbers. Choosing that scale is most of the game. The published state of the art for gradients (an estimator called MS-EDEN, from the Quartet II paper, arXiv:2601.22813) chooses it the safe way: scale so the block’s largest value lands exactly on 6, and nothing clips.

I found a scale rule that beats it by 14% on its own metric, on every real gradient tensor I tested. Then I rented two GPUs to watch the improvement show up in training loss, and it never did; four estimators landed within 0.09% of each other at 2.8B parameters. The number that explains both halves cost nothing to measure, and I measured it last, which is the embarrassing part. My batches were between 35 and 643 times too small for the difference to be visible. The batch size where it does become visible is the one frontier labs actually train at.

The code isn’t public, so this post carries the numbers instead. MS-EDEN here is my reimplementation (it reproduces the paper’s error figures, not its unbiasedness proof), and the training corpus is FineWeb-Edu, tokenized with GPT-2’s BPE.

The win, and where it comes from

A block of 16 gradient values is roughly bell-curved. Its largest value sits around 2 to 2.5 standard deviations. Scaling that value onto the grid’s top level spends the two coarsest levels (4 and 6) on numbers that almost never occur, and it starves the middle of the distribution where nearly everything lives.

The alternative is what anyone tuning a lossy codec would try: sweep 17 candidate scales per block and keep the one that minimizes mean squared error. Values above the top level clip; sacrificing the rare outlier buys precision for the common case. On synthetic data that’s worth +14.0% against MS-EDEN on Gaussian blocks and +25.0% on heavy-tailed ones. On 45 real weight-gradient tensors captured across three phases of training a small transformer, it wins on 45 of 45, mean +13.8% (worst tensor +9.6%, best +15.3%).

It stays hardware-valid, since the output is still the FP4 grid times one per-block scalar. And the caveat the experiment printed in its own output at the time matters for everything that follows: MSE is a proxy, and the training-loss impact was unvalidated.

The wall, then a landmine

Making the improvement provably unbiased failed, and the failure generalizes. On a fixed 16-level grid you can have a provably unbiased estimator (stochastic rounding, no clipping) or a low-error one (clipping), never both. The error reduction is the clipping, and stochastically rounding a clipped value is biased by construction. I later checked whether this was just NVFP4’s menu being bad. It isn’t: optimizing the palette itself moves the number but not the shape, and every 16-value palette I tested holds an unbiased-to-biased error ratio between 2.2× and 2.6×.

The standard escape is error feedback: keep the part you rounded off and add it back next step, Kahan summation applied to gradients. Each step stays biased, but the running sum doesn’t. That works exactly as advertised under SGD. On a coarse 3-level quantizer, a quantized-plus-feedback run lands on the full-precision trajectory to three digits (2.31e-5 against 2.33e-5, distance to the optimum).

Under Adam it’s a landmine. Adam rescales every update by a running second-moment estimate. The carried residual passes through that nonlinearity, and the guarantee quietly dies. Measured on the same harness, with orderings that hold at three sigma across seeds:

Error feedback under Adam was worse than no correction at all. Freezing the second moment (the fix the 1-bit Adam paper used for compressed communication) produced the best arm in that table, and I originally wrote that it repairs error feedback completely. The follow-up post ran the ablation I skipped, and the credit was misassigned: on this harness a frozen v helps just as much with no feedback, and with no quantization in the run at all. The fix was real; it just wasn’t fixing what I said it was. What survives is the landmine itself: don’t bolt error feedback onto Adam. I’d been running exactly that broken configuration on the GPUs without knowing it. At FP4’s error magnitudes it was too small to hurt, but the default was wrong.

Two GPUs that saw nothing

On a rented RTX 4090, a 0.5B-parameter GPT trained on 98M tokens. FP4 gradient quantization costs a real +2.14% in held-out loss against bf16, and my estimator differs from MS-EDEN by 0.01% (6.4442 against 6.4433, which is noise). The error-feedback arm never finished. Its fp32 residual buffer is a full parameter-sized copy, the card has 24 GB, and the log ends in a CUDA out-of-memory error trying to allocate 1.54 GiB with 773.69 MiB free.

So: a B200 with 192 GB, at $8.619 an hour, and a 2.8B-parameter model on 198M tokens, with the residual buffer halved to bf16 and activations checkpointed. All four arms ran, 2.06 hours of training time across them:

Everything sits inside a 0.09% band, which is seed noise. Even the 4090’s 2.14% bf16-versus-FP4 gap vanished, because 800 steps into a 2.8B model is nowhere near convergence. There was very little converged signal for quantization error to corrupt.

(An operational note, because it cost an hour of B200 billing: a relaunch command that ran pkill -f scale_experiment.py in the same shell that then launched scale_experiment.py matches its own command line and kills its own launcher. Every relaunch mechanism I tried “mysteriously” died until I read the exit code properly. Two other sessions on this machine have since hit the same trap. The fix is pgrep -f "patt[e]rn", where the brackets break the self-match.)

At this point the honest summary was: real on the proxy, invisible on the metric that matters, at every scale I could afford. I nearly wrote exactly that and stopped.

The number I should have measured first

The reason every run came back “within noise” has a formula. A gradient estimated from a batch of B tokens carries sampling noise that shrinks as 1/B. Quantization error doesn’t shrink with batch size at all. Divide one by the other and you get the batch size where a quantizer’s error, or the difference between two quantizers, pokes above the sampling-noise floor. Both quantities are measurable on a tiny model in minutes, on a laptop, for free.

Measured on real gradients (64-microbatch statistics, three training phases):

My runs used batches of 8,192 and 24,576 tokens. Depending on the run and the training phase, that is 35 to 643 times below the batch where the gap could have appeared in loss. The experiments were not measuring a small effect; they were structurally incapable of measuring any effect. It’s the benchmarking mistake in different clothes: evaluating a 1% optimization on a machine with 20% run-to-run variance, then concluding the optimization does nothing.

The crossover isn’t in exotic territory either. One to five million tokens per batch is the standard operating point of frontier-lab pretraining. Measured where I could afford to train, the difference is invisible; extrapolated to where the labs train, it sits right at the threshold of visibility. Two caveats travel with that. The crossover was measured on a 3M-parameter model, so treat it as an order of magnitude. And the confirming experiment (a ~124M-parameter model at a ~1.6M-token accumulated batch, where quantization overhead amortizes to nothing) is specced and smoke-tested but has not run. The prediction is a prediction, not a result.

One consolation prize from the same afternoon of cheap experiments: after the standard Hadamard rotation makes blocks Gaussian, plain evenly-spaced INT4 beats NVFP4’s float-style palette on both rounding modes (0.01445 against 0.01789 unbiased, 0.00621 against 0.00685 biased, normalized MSE). I first attached a condition to that, reasoning the uniform grid only wins after the rotation has removed the outliers the float spacing exists for. The follow-up that tested the condition on real tensors killed it: uniform INT4 wins on 41 of 45 gradient tensors with no rotation at all, because my heavy-tailed synthetic data had put the outliers somewhere real gradients don’t. The practical half survives untouched; integer 4-bit math predates Blackwell by years.

What generalizes

A null result is a claim about your noise floor at least as much as a claim about the effect. I ran the training experiment twice, on two GPUs, before measuring the floor once. The floor measurement took minutes, cost nothing, and retroactively explained both runs; it also converted “needs a converged frontier run to settle” into “needs one afternoon at a large batch.” The expensive experiments weren’t wrong, but every dollar of them was spent below the detection threshold, and the threshold was computable in advance.

I keep relearning the same discipline in different domains. Before paying to measure an effect, price the noise you’ll be measuring it through. The signal has to clear the floor, and the floor rarely announces itself. Mine was 35 to 643 times too high, and nothing in the loss curves even hinted at it.