NVIDIA Nemotron 3 Ultra: The Open Model That Optimised for Speed Instead of Leaderboards

NVIDIA released a 550-billion-parameter open-weight model on June 4, 2026 — and it doesn't top the intelligence leaderboard. It isn't close to Claude Fable 5. It scores below the leading Chinese open models.

It also serves over 400 output tokens per second, which is faster than a model four times smaller, and that single fact makes it the most economically interesting open-weight release of the year.

Most model coverage ranks by intelligence score and stops there. That framing systematically misreads what Nemotron 3 Ultra is for.

Introduction

There's a specific failure mode in how engineering teams evaluate models: they benchmark intelligence, pick the highest number they can afford, and discover six months later that inference cost and latency — not capability — are what actually constrain the product.

Nemotron 3 Ultra is built for teams who have already hit that wall.

NVIDIA released it as a fully open-weight model — pre-trained checkpoints, post-trained checkpoints, reward models, quantized variants, training datasets, and recipes, all public. It targets long-running agentic workflows: multi-step coding agents, enterprise document RAG, research automation, and orchestration pipelines where a model burns hundreds of turns before producing anything useful.

In that regime, tokens-per-second is not a vanity metric. It's the entire economics.

What Nemotron 3 Ultra Actually Is

Released: June 4, 2026 (announced at Computex 2026) Parameters: 550B total, 55B active per inference pass (Mixture-of-Experts) Context window: 1M tokens License: Open weights, available on Hugging Face and OpenRouter

The architecture is the story. From NVIDIA's official research page, Nemotron 3 Ultra uses a Mixture-of-Experts Hybrid Mamba-Attention architecture — not a pure transformer. It layers in four additional design choices:

1. LatentMoE — NVIDIA's expert-routing approach for improved accuracy per active parameter

2. MTP layers — Multi-Token Prediction for faster inference through native speculative decoding

3. Inference-time reasoning budget control — you can cap how much the model thinks per request

4. Pre-trained in NVFP4 — 4-bit precision from the start, not quantized after the fact

That last point is the one most coverage glosses over. Pre-training natively in NVFP4 rather than training in BF16 and quantizing afterward is why the speed numbers hold up without the usual quality collapse. Artificial Analysis measured the gap: BF16 scores 48.2 on the Intelligence Index versus 47.7 for NVFP4 — a 0.5 point difference for a 4× precision reduction.

Post-training used SFT, RL, and Multi-teacher On-Policy Distillation (MOPD) — distilling from several teacher models simultaneously rather than one.

The Benchmarks: An Honest Reading

Intelligence — where Nemotron loses

Artificial Analysis is direct about the positioning: Nemotron 3 Ultra scores 47.7, well ahead of the next strongest US open-weight models — Gemma 4 31B at 39.2, Nemotron 3 Super at 36.0, and gpt-oss-120b at 33.3 — but behind the Chinese-led open-weight frontier, with Kimi K2.6 at 53.9.

So: the most intelligent US open-weight model ever released, and not the most intelligent open-weight model. Both statements are true. Coverage that reports only the first is doing PR.

One more caveat worth knowing: GLM-5.2, released June 17, has since taken the open-weights Intelligence Index lead. In this market, "leading open model" has a shelf life measured in weeks.

Speed — where Nemotron wins decisively

NVIDIA's throughput comparisons are measured at an 8k token input / 64k token output setting — a long-output configuration that mirrors real agentic work rather than short chat turns.

The framing that matters, from Artificial Analysis: Nemotron 3 Ultra is served at over 400 output tokens per second — slightly faster than the typical serving speed of gpt-oss-120b despite being more than 4× larger, and with significantly greater intelligence.

A model 4× the size running faster than the smaller one is not an incremental gain. It's an architectural result.

Agentic performance — the Pareto frontier argument

This is the most methodologically interesting evaluation, and the one that best supports NVIDIA's design thesis.

Artificial Analysis ran Terminal-Bench v2.1 under restricted turn budgets — 10, 20, 50, and 100 turns — measuring both task success rate and wall-clock time per task. Their finding: Nemotron 3 Ultra completes tasks at a much faster pace than peers due to its high inference speed while scoring competitively on the benchmark, and it is the fastest across all turn limits while sitting on the Pareto frontier for performance versus time per task.

Read that carefully. Not "highest score." Best performance-per-unit-time at every turn budget tested.

For a chatbot, that distinction is academic. For an agent that runs 80 turns before returning a result, it's the difference between a 4-minute task and a 20-minute one.

NVIDIA's reported benchmark scores

From the Hugging Face model card, as catalogued by BenchLM:

Independent evaluation adds: GDPval-AA Elo of 1378 — a significant improvement over Nemotron 3 Super and approximately in line with DeepSeek V4 Flash.

Hallucination resistance — an underreported strength

Nemotron 3 Ultra scored 71% on AA-Omniscience Non-Hallucination, reflecting a low propensity to answer factual questions when it doesn't know the answer.

For RAG and enterprise document workflows, abstention behaviour is worth more than a few points of raw capability. A model that confidently fabricates a citation is worse than useless in a compliance context.

Token efficiency

Nemotron 3 Ultra uses roughly 1M fewer output tokens than Nemotron 3 Super to complete the full Intelligence Index evaluation — despite being larger and more capable. Less verbosity at higher capability is a direct cost reduction on every agentic run.

Where it genuinely underperforms

Two honest weak points:

Coding. Gemma 4 31B scores about 1 point higher than Nemotron 3 Ultra on the Artificial Analysis Coding Index (Terminal-Bench Hard and SciCode). A 31B model beating a 550B model on coding is a real gap, not a rounding error.

Frontier science. On CritPt, a benchmark of graduate-level physics research problems, Nemotron 3 Ultra reaches 3% — identical to Nemotron 3 Super. Scale delivered nothing here. If your workload is frontier scientific reasoning, this is not your model.

Benchmark coverage is still thin. BenchLM currently lists Nemotron 3 Ultra as unranked, citing insufficient non-vendor-generated benchmark coverage for a leaderboard position. Most published numbers still trace back to NVIDIA's own model card.

How It Compares: The Decision Matrix

The competitive claim from NVIDIA's launch coverage: Chinese open-weight models of comparable intelligence — DeepSeek V4 Pro and Kimi K2.6 — run at 50–100 tokens per second through their commercial APIs, making Nemotron 3 Ultra three to six times faster in practice.

That is the trade the entire model is built around: give up ~6 points of Intelligence Index, gain 3–6× throughput.

Where Nemotron 3 Ultra Is Actually the Right Choice

Use it when:

• You're running long-horizon agents where a single task consumes 50–100+ turns and latency compounds multiplicatively

• Inference cost is the binding constraint, not peak capability — NVIDIA claims up to 30% lower cost for agentic tasks

• You need self-hosted or air-gapped deployment — full open weights, no API dependency, no data leaving your infrastructure

• You're on Blackwell hardware — NVFP4 native pre-training means the quantized path is first-class, not an afterthought

• Long-context RAG at 1M tokens — NVIDIA reports it outperforms state-of-the-art open LLMs on RULER at 1M context

• Hallucination resistance matters — 71% non-hallucination on AA-Omniscience is strong for enterprise document work

Don't use it when:

• Peak intelligence is non-negotiable — Claude Opus 4.8 at 61 and Fable 5 above it are in a different tier

• Coding is the primary workload — Gemma 4 31B edges it on the Coding Index, and closed frontier models are far ahead

• You need frontier scientific reasoning — 3% on CritPt is a hard stop

• You have no infrastructure to self-host — a 550B model is not a laptop deployment, and the value proposition depends on controlling your own serving stack

The Strategic Read: Why NVIDIA Built This

Two things are happening at once, and both matter more than the benchmark table.

First: this is a hardware play. A model pre-trained in NVFP4, optimised for Blackwell, released free with full weights and training recipes — that isn't a model business. It's demand generation for GPUs. The better Nemotron runs on NVIDIA silicon relative to alternatives, the stronger the case for buying more of it. Giving away the model to sell the substrate is a coherent strategy, and it means Nemotron's roadmap will keep optimising for whatever NVIDIA's next architecture does well.

Second: this is a geopolitical positioning move. Launch coverage framed it explicitly as America's answer to the open-weight race with China, after DeepSeek and Kimi dominated open-weight leaderboards through most of 2026. The "most capable US open-weight model" framing is doing deliberate work.

Neither of these makes the model less useful. But they explain why the optimisation target is throughput rather than leaderboard position — and why you should evaluate it on that axis rather than the one the headlines use.

Key Takeaways

• Nemotron 3 Ultra is the most intelligent US open-weight model released to date (47.7 on AA Intelligence Index), and it is not the most intelligent open-weight model — Kimi K2.6 leads at 53.9, and GLM-5.2 has since taken the overall open-weight crown.

• Speed is the actual product. 400+ tokens/second, 5.9× the throughput of GLM-5.1 and 4.8× that of Kimi K2.6 at an 8k-in/64k-out configuration. A model 4× larger than gpt-oss-120b serving faster than it is an architectural achievement.

• The Pareto frontier result is the strongest evidence. Under turn-budgeted Terminal-Bench v2.1 testing, Nemotron 3 Ultra was fastest at every turn limit while remaining competitive on score — best performance-per-unit-time in the field.

• NVFP4 native pre-training is the enabling choice. Training in 4-bit from the start rather than quantizing afterward costs only 0.5 Intelligence Index points versus BF16 — an unusually clean trade.

• Know the weak spots. Coding trails a 31B model. CritPt physics performance is 3%, identical to the smaller Super variant. Independent benchmark coverage remains thin, with most numbers still sourced from NVIDIA's own model card.

• It's genuinely, fully open. Pre-trained, post-trained, and quantized checkpoints, the GenRM reward model, pre-training and post-training datasets, and full training recipes — all public. That completeness is rare, and it makes Nemotron 3 Ultra a serious base for teams doing their own fine-tuning.

Conclusion

Nemotron 3 Ultra will lose most head-to-head comparisons that lead with an intelligence score. It was not built to win those.

It was built for the specific economics of production agentic systems: hundreds of turns, long contexts, high volume, latency compounding at every step, and inference cost dominating the P&L. In that regime, a model that is 88% as intelligent and 5× faster is not a compromise — it's the correct answer.

The evaluation habit worth changing: stop asking "which model is smartest?" and start asking "which model delivers acceptable quality at the throughput and cost my workload actually requires?" Those are different questions, and the second one is the one that determines whether a system ships.

For teams building long-running agents on self-hosted infrastructure, Nemotron 3 Ultra deserves a serious benchmark against your real workload — not against a leaderboard.

References

Official NVIDIA sources:

• NVIDIA Nemotron 3 Ultra — research page

• Nemotron 3 Ultra Technical Report (PDF)

• Model checkpoints on Hugging Face — NVFP4, BF16, Base BF16, GenRM

• LatentMoE architecture

• NVIDIA Nemotron developer repository

Independent evaluation:

• Artificial Analysis — Nemotron 3 Ultra released — Intelligence Index, speed, Terminal-Bench Pareto analysis

• BenchLM — Nemotron 3 Ultra benchmark tracking

• Artificial Analysis Intelligence Index v4.1 methodology

Access:

• OpenRouter — Nemotron 3 Ultra 550B-A55B

Training datasets:

• Nemotron-Pretraining-Code-v3 — 173B tokens of GitHub code through September 30, 2025

• Nemotron-Pretraining-Legal-v1, Nemotron-Pretraining-Specialized-v1.2, Nemotron-Posttraining-v3

At Kynodex, we build and benchmark production agentic systems across open and closed models — including the routing layer that sends each task to the model with the right cost, latency, and quality profile. If your team is evaluating whether an open-weight model can carry your agent workload, talk to us.

Powered by Synscribe