Phi-mini-MoE

Claims and evidence

• Released 23 June 2025The model cards state no release date; the date is that of the SlimMoE paper, whose abstract says the models are publicly available and links both repositories. ● Primary source[1]Abstract (models made publicly available); v1 submission date

• Status AvailableWeights remain downloadable from Microsoft's Hugging Face organisation (checked 2026-10-01). ● Primary source[2]weights and licence (MIT)

• Derived from (distillation) Phi-3.5Compressed from Phi-3.5-MoE (an inline variant of the Phi-3.5 record) with the SlimMoE method of expert slimming and staged distillation, then post-trained with supervised fine-tuning and direct preference optimization. The model card says the starting point is the base model shared by Phi-3.5-MoE and GRIN-MoE. ● Primary source[1]Abstract [2]Model Summary

• Change · Size Compressed to 7.6B total and 2.4B active parameters, from 41.9B total and 6.6B active in Phi-3.5-MoE.Compared with Phi-3.5 ● Primary source[1]
• Change · Architecture Experts slimmed with the SlimMoE compression framework, with knowledge transferred through staged distillation.Compared with Phi-3.5 ● Primary source[1]Abstract

• Input text ● Primary source[2]Training, Model: Inputs
• Output text ● Primary source[2]Training, Model: Outputs
• Open weights Yes ● Primary source[2]licence: MIT [1]Abstract
• Context window 4k tokens ● Primary source[2]Training, Model: Context length
• Parameters 7.6B total, 2.4B active ● Primary source[2]Model Summary; Training, Model: Architecture [1]Abstract
• Access Open-weights download ● Primary source[2]Usage: Loading the model locally

Lineage

Variants

Related AI Radar coverage

AI Radar coverage starts in June 2026; no coverage linked yet.

All model releases from Microsoft on AI Radar →