The NCM-MLX project page has an animation. The same 128-token text appears in two panels, driven by a shared counter of model calls. The NCM panel shows all 128 positions from the first level and sharpens them level by level, and it’s done after 11 calls. The autoregressive baseline writes one token per call and needs 128.

It’s a striking picture, so here’s what it doesn’t say.

It counts calls, not time

An NCM call is bigger than an autoregressive call, and NCM adds a codec decode. So 11 versus 128 isn’t a speed-up of roughly 12 times, and the figure says so. The counter measures calls.

So how much faster is it?

The animation can’t tell you, and I haven’t measured wall-clock time for that TinyStories sample, so I won’t invent a number for it. What I have measured is a smaller task, TinyWorld, timed until the tokens were back on the CPU:

Batch	NCM	Autoregressive baseline	Roughly
1	4.5 ms	30.5 ms	7x faster
16	5.8 ms	22.9 ms	4x faster
64	10.8 ms	26.9 ms	2.5x faster
256	37.8 ms	49.8 ms	1.3x faster

Two caveats. The batch 1 baseline is noisy: an otherwise identical earlier run measured 19.8 ms, which would make it about 4x. And the advantage shrinks as the batch grows, because an NCM call is bigger and uses more memory. So the honest range on that small task is somewhere between 1.3x and 7x, depending on batch size and how busy the GPU is. It’s nowhere near 12x, and it says nothing yet about larger models. The full tables and the method are in the benchmarks doc on GitHub.

The baseline panel is a replay

The autoregressive side replays the same 128 tokens, one per call, using forced decoding. It isn’t an independent sample from that model. That’s fine for comparing the number of calls for identical output. It isn’t a quality comparison.

Quality still goes the other way

On text, the models trained in this project still lose clearly to the autoregressive baseline. GPT-2 large generative perplexity is 24.40 for the NCM against 11.48 for its baseline, and lower is better. The animation shows a trade, fewer and larger calls. The perplexity shows what that trade costs so far.

Why I’d still show it

Because the caption carries all of that. A figure with its caveats attached is a result. A figure without them is marketing. The sample is labelled too: it comes from the best model trained here, an 87M generator trained on more data, not the released checkpoint.

If you’re benchmarking something similar

• Say what the counter counts.
• Say which side is a replay.
• Put the quality number next to the speed number.
• Give the actual speed-up, with its range, so nobody has to guess.

The code, checkpoints and benchmarks are all in the NCM-MLX repository on GitHub.