BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization
BitsMoE targets ultra-low-bit quantization for MoE LLMs by allocating bits with spectral energy guidance, cutting accuracy loss while improving speed.
Intelligence analysis by GPT-5.4 Mini
The paper argues that standard pruning and coarse quantization waste capacity in MoE models, especially at very low bit widths. BitsMoE splits each layer into shared and expert-specific spectral parts, then uses an optimization step to assign bits where they matter most.
A big AI model can be split into many little workers. Only some workers are needed for each task, which saves effort, but all of them still take up room in memory.
BitsMoE is like packing a suitcase more carefully. It keeps the most shared parts safe and then squeezes the less important parts more tightly, so the model still works well even when it uses very few bits.
The paper says this helps the model stay smarter while getting smaller and faster. On one test model, it kept much more accuracy than an older method at the same tiny size.
Analysis
What the paper claims
Mixture-of-Experts LLMs activate only a subset of experts per token, which lowers compute, but the full set of expert weights still has to live in memory. The paper says existing compression methods do not work well in the ultra-low-bit range: pruning removes capacity permanently, and coarse quantization does not account for differences across experts or weight directions.
How BitsMoE works
BitsMoE uses an SVD-based decomposition for each MoE layer. It keeps the shared basis unquantized so common structure across experts is preserved, then treats the expert-specific spectral factors as the units to quantize. To choose bit widths, the method turns spectrum-wise mixed-precision quantization into an activation-aware reconstruction problem and solves an integer linear program to minimize estimated reconstruction loss under a fixed bit budget.
Reported results
The authors say experiments across multiple MoE LLMs show less downstream accuracy loss in ultra-low-bit regimes. On Qwen3-30B-A3B-Base at 2-bit quantization, they report 12.3× faster quantization, a 27.83-point gain in average accuracy, and 1.76× faster decoding than GPTQ. The paper also says the model and code are publicly available, which should make the approach easier to test and compare.
Key points
- BitsMoE is a quantization method for Mixture-of-Experts LLMs that focuses on ultra-low-bit settings.
- It keeps a shared spectral basis unquantized and quantizes expert-specific spectral factors more selectively.
- Bit widths are chosen with an activation-aware reconstruction objective solved as an integer linear program.
- The paper reports better accuracy retention than GPTQ in low-bit tests, including on Qwen3-30B-A3B-Base at 2 bits.
- The authors say the model and code are publicly available.
If the reported gains hold up, BitsMoE could make very large MoE models much cheaper to store and run at low bit widths. The public release of code and models should also make it easier for others to reproduce and extend the method.
The results are reported on a paper benchmark, so the method still needs broader validation across more models and real deployment settings. The integer-program step and SVD-based decomposition may also add complexity that not every production pipeline will want to absorb.
