NVIDIA is training a trillion-parameter open model, and capping the bill at $7bn
Reuters reported on 11 August that NVIDIA is developing Nemotron 4, an open-model family whose largest member is expected to exceed one trillion parameters — roughly twice Nemotron 3 Ultra. Training is unfinished, there is no release date, and employees suggested late autumn at the earliest.
The hardware vendor is now a model vendor with frontier ambitions, and the number attached to the ambition is public: The Information reports NVIDIA's cloud-compute budget for Nemotron development is capped at $7bn through fiscal 2028. A capped budget is a strategy statement. This is a line item, not a moonshot.
Scale first: Nemotron 3 Ultra runs 550 billion total parameters with 55 billion active through a mixture-of-experts design. Nemotron 4's flagship is expected to exceed a trillion total. The stated goal is parity with the best open models in the world, American and Chinese, and free to download and modify.
The logic is not subtle and does not need to be. Every open model that runs well on NVIDIA hardware enlarges the market for NVIDIA hardware, and every hour a developer spends inside the CUDA ecosystem is an hour not spent evaluating alternatives. Giving away weights to sell accelerators is a rational trade when the weights cost $7bn and the accelerators are the actual business.
It also lands the same week the company shipped an open router that sends each agent step to the cheapest model that can handle it — a tool whose entire premise is that the frontier model should be used as little as possible. NVIDIA is arming both sides of that argument, because it sells the picks either way.
Reuters via The Star — Nvidia is developing Nemotron 4 open-source models, The Information reports → · TechWire Asia — Nvidia reportedly builds 1-trillion-parameter Nemotron 4 AI model → · Technology.org — Nvidia Builds 1-Trillion-Parameter Nemotron 4 →