MiniMax teases upcoming M3 model with new sparse attention mechanism and 15.6X long-context response speed boost
MiniMax detailed its M2 engineering and teased M3, which it says uses sparse attention to speed up million-token responses by up to 15.6x.
Intelligence analysis by GPT-5.4 Mini
MiniMax is using a technical report on its M2 models to explain why it stuck with full attention for quality, while previewing M3 as a new design that could make ultra-long-context AI far cheaper and faster. The company says its sparse attention approach keeps reasoning strong while cutting the cost of long responses.
MiniMax is making a new kind of AI model that can read a lot more text much faster. It is like giving a student a huge stack of books, but letting the student find the important pages without rereading every single line.
The company says its older model was smart because it looked carefully at everything, but that way was slow and expensive. The new idea is to look in a smarter, more selective way so the model can still understand the story.
If it works, this could help AI helpers handle giant documents, big code projects, or long chats without getting sluggish. That would make them more useful for companies building new AI products.
Analysis
What MiniMax showed
MiniMax released a detailed report on the M2 family, including M2, M2.5, and M2.7, to explain the engineering choices behind its models. The report says the M2 stack uses a sparse mixture-of-experts decoder-only Transformer with 229.9 billion total parameters, while activating only 9.8 billion parameters per token across 256 experts.
The company also says it tuned routing with sigmoid gating and learnable expert-specific bias terms to reduce load-balancing issues without leaning on heavy auxiliary losses. Another key choice was to keep full multi-head attention with grouped query attention across all 62 layers, rather than relying on cheaper attention shortcuts.
Why M3 is the bigger signal
MiniMax’s argument is that earlier sub-quadratic approaches helped with cost but hurt reasoning, especially on long-context tasks and multi-hop reasoning. The company says it tested hybrid and windowed attention setups and found they degraded performance on harder long-document benchmarks.
That is why M3 is notable: MiniMax says the upcoming model will use a new sparse attention framework, called MiniMax Sparse Attention, that can reach up to 15.6x faster decoding at a million-token context. The company’s goal is to make ultra-long-context agent deployment economically viable.
For startups, the message is straightforward: long-context models are moving from a pure research problem toward an infrastructure question. If MiniMax’s approach works as advertised, it could lower the cost of agents that read huge documents, codebases, or archives while keeping enough reasoning quality to be useful.
Key points
- MiniMax published a technical report explaining the design of its M2 model family.
- The company says M2 uses a sparse mixture-of-experts Transformer with 229.9 billion total parameters.
- MiniMax says it tested cheaper attention methods but found they hurt long-context reasoning.
- M3 is being designed around a new sparse attention framework called MiniMax Sparse Attention.
- MiniMax claims the new approach can deliver up to 15.6x faster decoding at a million-token context.
