Xiaomi has just introduced Xiaomi Mimo, its first open-source AI large language model, designed for reasoning tasks.
Xiaomi MiMo
Xiaomi MiMo is a 7-billion-parameter model that excels in mathematical reasoning and code generation. It was developed by the company’s newly-formed Xiaomi Big Model Core Team and is said to match the performance of larger models like OpenAI’s 01-mini and Alibaba’s Qwen-32B-Preview.
The company believes that the reasoning effectiveness of MiMo, despite it being a smaller model, is driven by the base model’s potential, enabled through focused pre- and post-training strategies.
Pre-training
The reasoning ability of Xiaomi Mimo is built on an optimized pre-training process that includes improved data preprocessing pipelinne, enhanced text extraction tools, and multi-layered filtering. This was done to incrase the density of reasoning patterns.
The team compiled a dataset of 200 billion reasoning tokens and applied a three-stage data mixture strategy. The model was trained on 25 trillion tokens over three progressive training phases.
Post-Training Process
Meanwhile, the post-training stage used 130,000 mathematics and coding problems. The team also implemented a Test Difficulty Driven Reward system to address sparse rewards in complex tasks. There’s also Easy Data Re-Sampling for stable RL training on easier problems.
There’s also a Seamless Rollout Engine that cuts down GPU downtime, which increases training speed by 2.29x and a 1.96x boost in validation.
Xiaomi MiMo-7B comes in four versions:
- MiMo-7B-Base – the base model with strong reasoning potential
- MiMo-7B-RL-Zero – RL Model trained from the base
- MiMo-7B-SFT – a supervised fine-tuned model
- MiMo-7B-RL – RL model training from SFT
The MiMo-7B model series is open-source and accessible via Hugging Face. If you’re interested in the full technical report, you can check it out at GitHub.
Ram found his love and appreciation for writing in 2015 having started in the gaming and esports sphere for GG Network. He would then transition to focus more on the world of tech which has also began his journey into learning more about this world. That said though, he still has the mentality of "as long as it works" for his personal gadgets.
