A dense 30B with a million-token window, against a field that went sparse
Muse Glimmer pairs a 30B dense transformer with a ~1.8B vision encoder and a 1M-token context. Everyone else is shipping mixture-of-experts. Dense at this size is a deliberate bet.
Meta Muse Glimmer is a 30B dense multimodal transformer using grouped-query attention, paired with a roughly 1.8B ViT-G/14 perception encoder for image input, with a 1M-token context window and a stated focus on long-horizon coding and agentic knowledge work.
Dense is the notable word. The field's centre of gravity has moved decisively to mixture-of-experts — 975B total with 41B active, 276B with 12B, 675B with 41B. Sparse routing buys capability per unit of inference compute, and almost everyone has taken that trade.
A 30B dense model makes a different bet: every parameter participates in every token, which costs more per token at a given capability but removes the routing layer entirely. For long-horizon agentic work that may matter — MoE routing decisions are made per token, and consistency across a very long trajectory is a plausible place for a sparse model to behave unevenly in ways that are hard to debug.
It is also the size that fits comfortably on hardware people actually own, which is where the 1M context and the coding focus point. A dense 30B is a model you can run; a 975B MoE is a model you can call.
Whether the consistency argument holds is genuinely open — it is asserted more often than demonstrated, and the benchmarks that would separate the two do not really exist yet. But it is a real architectural fork in a month of releases that mostly differ by degree.
Augusto Digital — LLM News August 2026: Agent Breakthroughs & Price Cuts → · Augusto Digital — LLM News August 2026: Agent Breakthroughs & Price Cuts →