Benchmark database

Every measured run we have, stored individually rather than collapsed into one number per pairing. Where sources disagree — and on Strix Halo they disagree by nearly a factor of two — you see all of them.

Submit a benchmark

Measured runs

137

from published sources

Estimated pairings

1,980

generated by estimator_v1

Sources

GitHub, blogs, forums

Coverage

6.5%

of all pairings measured

Measured benchmark density is this database’s biggest weakness. 137 real runs against 1,980 estimated pairings means most intersection pages show a calculated range rather than a measurement. Those pages say so plainly. Benchmark contributions are the highest-value thing anyone could add.

Framework Desktop (Ryzen AI Max+ 395, 128 GB)

Model	Quant	Engine	Execution	Decode mode	Context	Decode	Prefill	Type	Date	Source
Qwen3-Coder 30B-A3B	`Q4_K_M`	llama.cpp	`resident`	standard	4K	52.00 t/s	—	Measured (community)	10 Feb 2026	blog
Qwen3.6 27B	`Q4_K_M`	LM Studio	`resident`	standard	4K	11.50 t/s	—	Measured (community)	10 Feb 2026	blog
gpt-oss-120b	`MXFP4`	LM Studio	`resident`	standard	4K	30.00 t/s	—	Measured (community)	10 Feb 2026	blog
gpt-oss-120b	`MXFP4`	llama.cpp	`resident`	standard	4K	55.00 t/s	—	Measured (community)	15 Mar 2026	blog
gpt-oss-120b	`MXFP4`	llama.cpp	`resident`	standard	4K	31.00 t/s	—	Measured (community)	2 Apr 2026	blog
gpt-oss-120b	`MXFP4`	llama.cpp	`resident`	standard	4K	34.00 t/s	—	Measured (community)	11 May 2026	blog

Reported as a 35B-class MoE with 3B active exceeding 50 t/s with a draft model.
A dense 27B on Strix Halo: roughly a third the speed of a 120B MoE on the same box. The clearest single illustration of why active parameter count matters more than total size for decode.
LM Studio with the ROCm llama.cpp runtime.
Substantially higher than other Strix Halo reports for the same model. Retained deliberately: the disagreement between sources is real information and is reflected in the confidence score rather than averaged away.

GMKtec EVO-X2 (Ryzen AI Max+ 395, 128 GB)

Model	Quant	Engine	Execution	Decode mode	Context	Decode	Prefill	Type	Date	Source
gpt-oss-120b	`MXFP4`	llama.cpp	`resident`	standard	4K	32.00 t/s	—	Measured (community)	20 Apr 2026	github

Same silicon as the Framework Desktop; results track within noise, which is the expected outcome and a useful sanity check on our hardware model.

NVIDIA DGX Spark 128 GB

Model	Quant	Engine	Execution	Decode mode	Context	Decode	Prefill	Type	Date	Source
DeepSeek-V4 Flash	`DS4 Q2 (mixed)`	DwarfStar bd66c402070042bf0a79ad6ece8242de4c93680c	`resident`	standard	2K	18.05 t/s	826 t/s	Measured (first-party)	6 Sept 2026	github
DeepSeek-V4 Flash	`DS4 Q2 (mixed)`	DwarfStar bd66c402070042bf0a79ad6ece8242de4c93680c	`resident`	standard	16K	15.10 t/s	872 t/s	Measured (first-party)	6 Sept 2026	github
DeepSeek-V4 Flash	`DS4 Q2 (mixed)`	DwarfStar bd66c402070042bf0a79ad6ece8242de4c93680c	`resident`	standard	32K	14.43 t/s	856 t/s	Measured (first-party)	6 Sept 2026	github
DeepSeek-V4 Flash	`DS4 Q2 (mixed)`	DwarfStar bd66c402070042bf0a79ad6ece8242de4c93680c	`resident`	standard	64K	13.84 t/s	823 t/s	Measured (first-party)	6 Sept 2026	github
DeepSeek-V4.1 Flash	`DS4 V4.1 Q2 (SSD)`	DwarfStar 6e4c285a92a1de11863784b748f196b5d4b88a48	`ssd streamed`	standard	1K	9.63 t/s	24 t/s	Measured (first-party)	12 Sept 2026	github
DeepSeek-V4.1 Flash	`DS4 V4.1 Q2 (SSD)`	DwarfStar 6e4c285a92a1de11863784b748f196b5d4b88a48	`ssd streamed`	standard	4K	9.15 t/s	46 t/s	Measured (first-party)	12 Sept 2026	github
DeepSeek-V4.1 Flash	`DS4 V4.1 Q2 (SSD)`	DwarfStar 6e4c285a92a1de11863784b748f196b5d4b88a48	`ssd streamed`	standard	16K	8.22 t/s	40 t/s	Measured (first-party)	12 Sept 2026	github
gpt-oss-120b	`MXFP4`	llama.cpp b7db35a7	`resident`	standard	0	60.57 t/s	1,956 t/s	Measured (community)	20 Oct 2025	github
gpt-oss-120b	`MXFP4`	llama.cpp	`resident`	standard	0	58.70 t/s	1,723 t/s	Measured (independent)	25 Oct 2025	blog
gpt-oss-120b	`MXFP4`	llama.cpp b7db35a7	`resident`	standard	4K	54.14 t/s	1,637 t/s	Measured (community)	20 Oct 2025	github
gpt-oss-120b	`MXFP4`	llama.cpp b7db35a7	`resident`	standard	8K	51.54 t/s	1,512 t/s	Measured (community)	20 Oct 2025	github
gpt-oss-120b	`MXFP4`	llama.cpp b7db35a7	`resident`	standard	16K	47.45 t/s	1,307 t/s	Measured (community)	20 Oct 2025	github
gpt-oss-120b	`MXFP4`	llama.cpp b7db35a7	`resident`	standard	32K	40.55 t/s	1,027 t/s	Measured (community)	20 Oct 2025	github

Fully resident DeepSeek V4 Flash 0731 DS4 Q2. ds4-bench used 2,048-token prefill steps, 128 greedy generation tokens and no speculative decoding.
Same fully resident DwarfStar context sweep at 16K occupied context.
Same fully resident DwarfStar context sweep at 32K occupied context.
Same fully resident DwarfStar context sweep at 64K occupied context.
Median of three complete DwarfStar sweeps. DeepSeek V4.1 Flash Q2 used SSD streaming, automatic expert-cache sizing (80.17 GiB of experts), disk-only Engram tables, 32K allocated context and no speculative decoding.
Median of the same three complete SSD-streaming sweeps at the 4K context frontier; 3,072 tokens were added after the 1K frontier.
Median of the same three complete SSD-streaming sweeps at the 16K context frontier; 12,288 tokens were added after the 4K frontier.
llama-bench at zero context depth: the best case, and the number most often quoted in marketing. Reported as 60.57 +/- 0.25.
Independent reproduction. Slightly below the llama.cpp thread figure, which is normal for a different kernel build.
Same run at 4K context depth. Reported as 54.14 +/- 0.08.
8K context depth. Reported as 51.54 +/- 0.14.
16K context depth. Reported as 47.45 +/- 0.08.
32K context depth — decode has fallen 33% from the zero-context figure. This is why the site buckets by context rather than quoting one number.

RTX 3090 workstation (Ryzen 9 7950X, 64 GB)

Model	Quant	Engine	Execution	Decode mode	Context	Decode	Prefill	Type	Date
Qwen3.8 27B	`Q5_K_M`	llama.cpp	`resident`	`MTP`	1K	74.88 t/s	—	Measured (community)	25 Aug 2026
Qwen3.8 27B	`Q5_K_M`	llama.cpp	`resident`	`MTP`	16K	70.15 t/s	—	Measured (community)	25 Aug 2026
Qwen3.8 27B	`Q5_K_M`	llama.cpp	`resident`	`MTP`	63K	56.58 t/s	—	Measured (community)	25 Aug 2026
Qwen3.8 27B	`Q5_K_M`	llama.cpp	`resident`	`MTP`	90K	42.71 t/s	—	Measured (community)	25 Aug 2026

Approximately 1K tokens of occupied context. Qwen3.8-27B-heretic-ara Q5_K_M GGUF in llama.cpp with MTP enabled; Pi was the agent harness. This is output rate with accepted draft tokens, not ordinary one-token-at-a-time decode.
Approximately 16K tokens of occupied context. Same MTP-enabled run and hardware as the 1K observation.
Approximately 63K tokens of occupied context. Same MTP-enabled run and hardware as the 1K observation.
Approximately 90K tokens of occupied context. Same MTP-enabled run and hardware as the 1K observation; the author capped the configured window at 96K to fit MTP in 24 GB VRAM.

RTX 5090 workstation (1x 32 GB)

Model	Quant	Engine	Execution	Decode mode	Context	Decode	Prefill	Type	Date	Source
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.54 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.17 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.05 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.44 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.97 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.33 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.27 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.24 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	64.71 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	66.74 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	66.70 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.80 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.73 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	68.77 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	66.00 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.17 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.30 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.67 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.49 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.37 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.36 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.39 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.05 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.67 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.37 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.16 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.19 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	66.19 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.37 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	66.39 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	66.75 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.31 t/s	—	Measured (independent)	1 Jul 2026	docs
Gemma 4 31B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.70 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3 30B-A3B	`Q4_K_M`	llama.cpp	`resident`	standard	0	352.06 t/s	—	Measured (community)	30 Nov 2025	github
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	71.26 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	68.97 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.28 t/s	—	Measured (independent)	2 May 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.68 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.68 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	68.61 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	71.81 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.55 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.49 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.27 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.05 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.34 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	45.92 t/s	—	Measured (independent)	1 May 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.31 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.84 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.70 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.56 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	66.28 t/s	—	Measured (independent)	27 Apr 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.90 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.54 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.48 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.59 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.84 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	39.61 t/s	—	Measured (independent)	1 May 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.60 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	73.56 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.84 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.89 t/s	—	Measured (independent)	2 May 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	70.98 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.61 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	70.89 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.32 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	70.07 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	73.96 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	71.99 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.15 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	71.50 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.63 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	73.76 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.18 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.44 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.90 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	47.75 t/s	—	Measured (independent)	1 May 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.57 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.45 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	71.09 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.65 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.48 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.79 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.39 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.23 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.95 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.53 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	73.26 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	67.86 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	68.00 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.19 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	73.20 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.56 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	68.94 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.62 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.56 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	70.95 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.79 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.36 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	68.82 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.48 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	69.56 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	73.31 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	70.04 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	70.37 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	71.92 t/s	—	Measured (independent)	30 Jun 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	68.93 t/s	—	Measured (independent)	1 Jul 2026	docs
Qwen3.6 27B	`Q4_K_M`	llama.cpp	`resident`	standard	128	72.38 t/s	—	Measured (independent)	1 Jul 2026	docs
Ternary Bonsai 2 27B	`PQ2_0`	Prism ML llama.cpp fork	`resident`	standard	0	129.90 t/s	3,893 t/s	Vendor-reported	—	vendor
Ternary Bonsai 2 27B	`PTQ1_0`	Prism ML llama.cpp fork	`resident`	standard	0	120.50 t/s	1,805 t/s	Vendor-reported	—	vendor
gpt-oss-20b	`MXFP4`	llama.cpp	`resident`	standard	0	419.14 t/s	—	Measured (community)	30 Nov 2025	github

Imported from canonical llm-speed run r_-_ag61ig4f_; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_-dxzsj2j-y_; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_0dfevy9f4gh; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_0igp5bsgx_7; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_0ztzrxt5qw2; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_1ofpvf4m7p5; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_23fireoga9y; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_4fi6u0-u2ih; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_608esz63ib8; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_616apthaana; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r__679a-4h44q; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_dee_mcm1ga0; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_ea3vckhh-cn; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_epzfs8k8ohh; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_f-uojs7vckb; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_f1wlemwip05; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_jkn4ltg393p; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_lrxodemb-w0; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_lzv-y6_-n0b; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_moawmhigq5d; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_n65ex7zl6ts; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_n6su7ptrulh; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_onvh-dpmck-; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_pa7nuerd9tl; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_plujbqnef08; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_pnk9rze10h7; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_q-it_4s9z4p; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_s0-yb8j14ys; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_uut6m_v6ui9; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_wppiqxyk9iw; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_wtd90zbbusq; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_wwq2l7lmy6d; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_yz3q11pa049; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
llama-bench tg128, Qwen3 30B-A3B Q4_K_M (17.28 GiB), full GPU offload, flash attention enabled and GGML_CUDA_GRAPH_OPT=1. Reported as 352.06 +/- 1.76 t/s. The source does not pin a llama.cpp commit, which caps confidence.
Imported from canonical llm-speed run r_-1k-8qr_v8a; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_-cpk_tabogc; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_1pww-w7p8sd; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_2673sv7x5m4; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_2g-h0epeovp; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_2o9kb8hy9uk; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_3hda1iz7yd8; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_3n3hnvdgwnx; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_3un_x00tm0t; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_432qs45nio9; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_4fxpo_ja16m; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_4puowxcjcfu; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_4u7250hj28o; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_4upad7ubcpi; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_5-8ewf89dyr; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_5b-e5ortgcv; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_5m7gbxjpw_c; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_79bwm4mq_4l; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_7snxv0llk6f; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_7u98v6--uli; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_83zbwf7c17p; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_8qwzo79cf0s; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_9ap2a34-le5; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r__b89kg2iica; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_aj808r0dw53; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_az687c4xasr; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_bgylub2qqr-; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_bqsunbd6xa8; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_carzhcoe-mi; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_e7_858msfny; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_ea1db-lhv0r; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_ercjdbdw2gi; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_es8a88t0ez0; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_f7ulllqu4vk; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_fo83sbnxax4; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_g37y03ugmpk; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_gipbd9a8wrn; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_j0xki0x2kn2; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_j1cr6asrnn6; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_j1t3p2tae3a; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_jbit-ru039h; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_jrwk2oj1el-; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_kj4fh_mmzj9; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_l-wg5a6o-vg; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_lblt3fsiqzb; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_mj85rtghwmg; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_mqx-i-thqtq; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_p01v4lc4c11; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_p307g-1cdkl; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_p3m8a1lj4kg; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_q0wcxrit2bz; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_q5xsw1p-7f8; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_qdm3mkehpyx; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_qe0el0pkw93; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_qhp37dyi9n1; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_qiq8q_cqfk5; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_qlcfyt0mlv6; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_qlo6j6lnnh3; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_qrpvh5hd9km; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_qsvrisrvp-e; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_th_ounxmvpx; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_u4wa_y_y3vt; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_u9eqyrsfrgt; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_uhv-sxt4h97; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_uvy0zplf-sh; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_v05axrj3dti; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_v2vrkr4uah1; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_w482rf72v6z; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_wax_x2ryqhk; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_wmexo75dt0o; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_wowv__0b5s6; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_wqhqzea_7xy; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_xa08qutcmkm; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Imported from canonical llm-speed run r_yluotk909p8; chat-short batch 1. Actual prompt tokens reported by the backend: 0.
Publisher's RTX 5090 GPU result, associated with ModelRigs' representative single-5090 workstation. TG128 decode and PP512 prefill are separate llama-bench tests at depth zero; each is a reported mean. Exact host configuration, binary revision, and test date are not specified on the pinned card. No vision projector.
Publisher's RTX 5090 GPU result, associated with ModelRigs' representative single-5090 workstation. TG128 decode and PP512 prefill are separate llama-bench tests at depth zero; each is a reported mean. Exact host configuration, binary revision, and test date are not specified on the pinned card. No vision projector.
llama-bench tg128, native gpt-oss 20B MXFP4 (11.27 GiB), full GPU offload, flash attention enabled and GGML_CUDA_GRAPH_OPT=1. Reported as 419.14 +/- 0.89 t/s. The source does not pin a llama.cpp commit, which caps confidence.

Used RTX 4090 workstation (24 GB)

Model	Quant	Engine	Execution	Decode mode	Context	Decode	Prefill	Type	Date	Source
Ternary Bonsai 2 27B	`PQ2_0`	Prism ML llama.cpp fork	`resident`	standard	0	81.20 t/s	3,124 t/s	Vendor-reported	—	vendor
Ternary Bonsai 2 27B	`PTQ1_0`	Prism ML llama.cpp fork	`resident`	standard	0	91.10 t/s	1,645 t/s	Vendor-reported	—	vendor

Publisher's RTX 4090 GPU result, associated with ModelRigs' representative used-4090 workstation. TG128 decode and PP512 prefill are separate llama-bench tests at depth zero; each is a reported mean. Exact host configuration, binary revision, and test date are not specified on the pinned card. No vision projector.
Publisher's RTX 4090 GPU result, associated with ModelRigs' representative used-4090 workstation. TG128 decode and PP512 prefill are separate llama-bench tests at depth zero; each is a reported mean. Exact host configuration, binary revision, and test date are not specified on the pinned card. No vision projector.

Sources

Title	Type	Author	Accessed	Notes
AMD Ryzen AI Max+ 395 (Strix Halo) for Local LLMs in 2026	blog	RunAIHome	6 Sept 2026	Community review with throughput figures for MoE models on Strix Halo.
GPT-OSS 120B & 20B on AMD Ryzen AI Max+ 395	blog	akehir	6 Sept 2026	Kubernetes-hosted llama.cpp on Strix Halo. Reports a materially higher figure than other Strix Halo sources, which is preserved rather than smoothed away.
Local LLM Speed Test: GPT-OSS, Qwen3.6 and Hermes on 128GB Unified Memory	blog	MindStudio	6 Sept 2026	LM Studio with the ROCm llama.cpp runtime on a 128 GB Ryzen AI Max+ 395.
Strix Halo Tokens Per Second 2026: Real Speed, Dense vs MoE	blog	DataHardware	6 Sept 2026	Aggregated Strix Halo measurements across dense and MoE models.
llama.cpp on NVIDIA DGX Spark — Benchmarks	blog	JetsonHacks	6 Sept 2026	Independent reproduction of the llama.cpp DGX Spark benchmarks.
llm-speed open benchmark dataset	docs	llm-speed contributors	21 Sept 2026	CC BY 4.0. Imported rows link to their canonical per-run pages and retain the exact source run ID, hardware fingerprint, model identifier, quantization, backend metadata and suite version.
localai.computer measured benchmark dataset	docs	localai.computer	21 Sept 2026	CC BY 4.0. LocalAI provenance and each original upstream URL are retained. Legacy rows have unknown context; generatedTokens is output length and is never treated as context.
Strix Halo (Ryzen AI Max+ 395) LLM Benchmark Results	forum	Level1Techs forum	6 Sept 2026	Community benchmark thread with a wide spread of results depending on backend (Vulkan vs ROCm).
AMD Strix Halo — Backend Benchmarks (Grid View)	github	kyuz0	6 Sept 2026	Systematic backend comparison grid for Strix Halo. One of the few Strix Halo sources with full method documentation.
DwarfStar DeepSeek V4 Flash Q2 context sweep on DGX Spark	github	antirez / DwarfStar contributors	13 Sept 2026	Runtime-maintainer CSV produced by ds4-bench with 2,048-token prefill steps and 128 greedy generation tokens. The imported points and artifact instructions are pinned to DwarfStar commit bd66c402070042bf0a79ad6ece8242de4c93680c.
DwarfStar DeepSeek V4.1 Flash Q2 SSD-streaming measurements on DGX Spark	github	antirez / DwarfStar contributors	14 Sept 2026	Runtime-maintainer QA record for the 341 GiB DwarfStar Q2 artifact on one 128 GB DGX Spark. Three complete sweeps used automatic expert-cache sizing, 32,768 allocated context and 128 generated tokens per frontier. A separate two-Spark RoCE result is documented upstream but is not assigned to the single-Spark configuration.
Optimizing Token Generation in llama.cpp's CUDA Backend	github	llama.cpp contributors	19 Oct 2026	Published llama-bench decode results for Qwen3 30B-A3B Q4_K_M and gpt-oss 20B MXFP4 on RTX 5090. The source gives the command, backend flags, model sizes and repeated-result dispersion, but not a pinned llama.cpp commit, so confidence remains community-level.
Performance of llama.cpp on Apple Silicon M-series	github	llama.cpp contributors	6 Sept 2026	The long-running Apple Silicon llama-bench collection. Uses Llama 7B as a fixed reference model across chips, which makes it excellent for calibrating a hardware bandwidth coefficient and poor for predicting any specific modern model.
Performance of llama.cpp on NVIDIA CUDA	github	llama.cpp contributors	19 Oct 2026	Maintained llama-bench CUDA scoreboard. The imported RTX 5090 and RTX PRO 6000 Llama 7B Q4_0 rows retain their individual llama.cpp commits and are calibration-only because the reference model is not in the catalogue.
Performance of llama.cpp on NVIDIA DGX Spark	github	llama.cpp contributors	6 Sept 2026	Maintained benchmark thread with llama-bench output at several context depths. Build 7db35a7. Numbers include the reported standard deviation, which is why the confidence weight is high for a community source.
Performance of llama.cpp with Vulkan	github	llama.cpp contributors	19 Oct 2026	Maintained Vulkan llama-bench scoreboard with a standardized Llama 7B Q4_0 workload. The R9700 row is used only for hardware calibration.
Bonsai 2 27B GGUF cross-platform throughput	vendor	Prism ML	18 Sept 2026	Publisher-reported llama-bench means for custom Prism ML llama.cpp kernels. TG128 generates 128 tokens and PP512 processes 512 input tokens; these are separate depth-zero, batch-one tests with no vision projector. Card revision 6ed5e12bf84b7a63069882c91dd9e9218647d17b. Exact benchmark binary revision and test date are not stated on the card.