Phi-3.5 MoE 42B-A6.6B
Phi-3.5 MoE 42B-A6.6B canonical official release.
MicrosoftMixture of expertsgeneral0 machines can run it
Total parameters
41.873B
Official spec sets memory need
Active parameters
6.6B
Official spec sets decode speed
Context
128K
Official spec tokens
4-bit weights
23 GB
Calculated before KV cache
This is a sparse mixture of experts. All 41.873B of weights must be available to the runtime, but only 6.6B are read per generated token. Ordinary runtimes keep the full quantized model in memory; a runtime-specific SSD-streaming artifact can retain a smaller working set and fetch expert data on demand, trading speed for capacity.
Model specification
Publisher
Microsoft Official spec
Architecture
Sparse mixture of experts Official spec
Total parameters
41.873B Official spec
Active parameters
6.6B per token Official spec
Context length
131,072 tokens Official spec
Attention ⓘ
GQA · 8 KV heads × 128 dim × 32 layers Official spec
Licence
MIT Official spec
Specification confidence ⓘ
High · verified 2026-09-26 Official spec source
Released
20 Aug 2024 Official spec
Official source
Quantizations and memory
| Quantization | Format | Bits/weight | Resident | Download | Quality kept |
|---|---|---|---|---|---|
MLX 4-bit | mlx | 4.5 | 22.8 GB | 22.8 GB | 98.0% |
Q4_K_Mdefault | gguf | 4.85 | 24.4 GB | 24.4 GB | 98.5% |
Q5_K_M | gguf | 5.69 | 28.6 GB | 28.6 GB | 99.3% |
Q8_0 | gguf | 8.5 | 42.3 GB | 42.3 GB | 99.9% |
BF16 | safetensors | 16 | 78.0 GB | 78.0 GB | 100.0% |
Generic weight sizes are computed from the parameter count and bits per weight plus a format-specific overhead. Runtime-specific artifacts use their published resident and download footprints; streamed models can therefore require much more disk than memory. Quality retention is an assumption, not a measured evaluation.
Coding & quality benchmarksCompare coding results →
No published benchmark results have been imported for this model yet. This is missing evidence, not a score of zero.
We show source metrics rather than deriving one opaque quality number. Different benchmarks measure genuinely different things, and collapsing them into a single score would hide exactly the disagreements worth seeing.
Recommendations
Cheapest that can run it
Nothing in the database qualifies.
Lowest purchase price among configurations where the model fits at some quantization in our catalogue. Speed is not considered.
Cheapest above 20 t/s
Nothing in the database qualifies.
20 tokens/second is roughly the point at which generation keeps pace with reading. Below it, interactive use feels like waiting.
Cheapest above 40 t/s
Nothing in the database qualifies.
40 tokens/second is the threshold most people describe as comfortable for coding agents, where output arrives faster than you can review it.
Fastest with real measurements
Nothing in the database qualifies.
Highest throughput among configurations with an actual published measurement rather than our estimate.
Best throughput per purchase-price unit
Nothing in the database qualifies.
Highest decode tokens/second per 1,000 units of the displayed purchase currency. Ignores running costs and resale — see the economics section for the full picture.
Hardware that runs Phi-3.5 MoE 42B-A6.6B0 measured, 0 estimated
0 of 0 rows
| Hardware↕ | Quant | Memory↕ | Decode▼ | Prefill↕ | Context | Price↕ | Fit | Confidence↕ |
|---|---|---|---|---|---|---|---|---|
| Nothing matches these filters. | ||||||||
Similar models
Mixtral 8x7B
Mistral AI · 46.703B (13B active) · Apache 2.0
Qwen3 30B-A3B
Alibaba Qwen · 30.532B (3.3B active) · Apache 2.0
Gemma 4 26B-A4B
Google DeepMind · 25.806B (3.8B active) · Gemma Terms of Use
gpt-oss-20b
OpenAI · 20.915B (3.6B active) · Apache 2.0
Command R 35B
Cohere For AI · 34.981B · CC-BY-NC-4.0
Yi 1.5 34B
01.AI · 34.389B · Apache 2.0