Up to 3.09x faster

← Back to the write-up and video on hijoseromero.com

Multi-Token Prediction (MTP) on Qwen 3.8-27B at full BF16 precision, measured on my own desk. One model, three very different prompts, four MTP settings, and the whole thing run twice: once in thinking mode with the model card's recommended sampling, once in a clean-room configuration built for exact matching. Every raw output opens in a new tab via the audit links, and every leg carries its run timestamp.

Data collected 2026-08-22 23:15 → 2026-08-23 17:35 local time · page generated 2026-08-25 23:51 · every number measured on the machine described at the bottom, logs retained.

The two experiments, and why there are two

Shared by every leg of both experiments: Qwen 3.8-27B BF16 (54.7 GB GGUF, Unsloth repack) · seed 42 on every request · one prompt, one shot, no retries · --parallel 1 · tools disabled · 65,536-token context · no output cap (60k safety ceiling) · fresh server launch per MTP setting · llama.cpp build 10472 (Unsloth) · no agent harness anywhere, raw API calls only · speeds and acceptance scraped from llama.cpp's own per-request timing log · peak memory sampled every 5 s.

Experiment 1: thinking ON, model-card sampling

"Guesses accepted" is the honest column: MTP lets a tiny helper guess a few tokens ahead, and the model only keeps guesses it fully agrees with. High acceptance = big speedup. Watch it collapse from the mechanical Table to the free-form Essay.

The Table 60 rows of formula-driven inventory data, every cell checkable

Generate a markdown table for a warehouse inventory with EXACTLY 60 product rows plus a final TOTAL row.

Columns: Row | SKU | Product | Unit Price | Stock | Line Value

Rules, follow them exactly:
- Row numbers run 1 through 60.
- SKU for row N is "WH-" followed by N padded to 4 digits (row 7 = WH-0007).
- Product name for row N is "Widget Model N".
- Unit Price for row N is 10.00 + (N * 0.25), shown with 2 decimals.
- Stock for row N is 500 - (3 * N).
- Line Value is Unit Price * Stock, shown with 2 decimals.
- The final TOTAL row shows the sum of the Stock column and the sum of the Line Value column.

Do all arithmetic silently. Your answer must be ONLY the finished markdown table: no calculation steps, no explanations, no text before or after it.

Legs of this test completed 2026-08-23 02:55 → 2026-08-23 03:33 (local time; each row above carries its own stamp).

The Boilerplate a five-resource FastAPI back end, repetitive by design

Write a complete, single-file FastAPI application implementing a bookstore back end with five resources: books, authors, publishers, genres, and customers.

Requirements:
- Each resource gets the identical five endpoints: list all, get by id, create, update, delete.
- Each resource gets its own Pydantic model with at least four typed fields.
- Storage is a plain in-memory dict per resource; ids are integers assigned from a counter.
- Every function has a type-hinted signature and a one-line docstring.
- No authentication, no database, no comments about what you might do differently.

Output only the Python code in a single fenced block, no commentary before or after.

Legs of this test completed 2026-08-22 23:39 → 2026-08-23 00:37 (local time; each row above carries its own stamp).

The Essay 800 words of unpredictable prose, the draft head's worst case

Write a reflective essay of roughly 800 words on what beekeeping can teach a software engineer about database indexing. Draw at least three specific parallels, ground each one in a concrete detail of how bees or hives actually work, and end with one practical lesson a working engineer could apply this week. Avoid bullet points; write flowing prose.

Legs of this test completed 2026-08-22 23:49 → 2026-08-23 00:49 (local time; each row above carries its own stamp).

Did it change the output?

Three views of the same question. First the verdict matrix: every pairing, thinking and answer judged separately. Then who-matched-whom: the legs grouped by file hash, so the pattern is visible at a glance. Then the highlights.

✅ every byte matched · 🟢 the final ANSWER matched even though the thinking transcript took a different path (different reasoning, same destination) · 🟠 thinking matched but the answer forked (the split happened while writing the answer) · 🟡 same words, only formatting/casing differs · ❌ the answers genuinely differ (equally valid, see the side-by-side splits below). Left columns: each depth against MTP off. Right columns: each depth against the previous depth, which shows whether the MTP legs agree with EACH OTHER.

Legs joined by = produced byte-identical files (first 8 hex digits of the SHA-256 shown). They can be recomputed from the audit links.

Where exactly they split

Skim this. One card per distinct divergence: the last stretch both legs shared, then what each wrote next. Depths that wrote the same continuation are merged into one card. Identical pairs are stated in one line.

The Table

✅ identical (answer + thinking) MTP off vs depth 2

What about the other columns? I'll put dashes or leave them blank. Let me just put "TOTAL" in the Product colu…

For other columns in the TOTAL row, I'll leave them blank or put a dash.

Actually, re-reading the instruction…

What about the other columns? I'll put dashes or leave them blank. Let me just put "TOTAL" in the Product column and the sums in Stock and Line Value columns.

Let me format this properly.
 ⟨thinking ends, answer begins⟩ | Row | SKU | Product | Unit Price | Stock | Line Value |
|-----|-----…

For other columns in the TOTAL row, I'll leave them blank or put a dash.

Actually, re-reading the instructions: "The final TOTAL row shows the sum of the Stock column and the sum of the Line Value column." So I'll put the values in those columns and leave the…

The Boilerplate

Endpoints ───────────────────────────────────────────────────────────

@app.get("/books", response_model=List[…

─────────────────────────────────────────────────────────────────────

@app.get("/books", response_model=List[…

Endpoints ───────────────────────────────────────────────────────────

@app.get("/books", response_model=List[Book])
def list_books() -> List[Book]:
    """Return all books."""
    return list(books_store.values())


@app.get("/books/{book_id}", response_model…

─────────────────────────────────────────────────────────────────────

@app.get("/books", response_model=List[Book])
def list_books() -> List[Book]:
    """Return all books."""
    return list(books_store.values())


@app.get("/books/{book_id}", response_model…

The Essay

someone who has actually kept bees or spent time with hives, reflecting on their engineering work.

Let me dra…

like an engineer who has taken a beekeeping class or observed a hive and had a "click" moment.

Let me draft t…

someone who has actually kept bees or spent time with hives, reflecting on their engineering work.

Let me draft this and aim for roughly 800 words. Let me count carefully.

Let me write it now in flowing prose, reflective, no bullets.
 ⟨thinking ends, answer begins⟩ I have kept bees for th…

like an engineer who has taken a beekeeping class or observed a hive and had a "click" moment.

Let me draft this and count words roughly. I'll aim for 780-830.

Let me write it now in flowing prose, no bullet points.
 ⟨thinking ends, answer begins⟩ There is a particular Tuesday in late Apr…

waggle dance / navigation**: Bees use the waggle dance to communicate the location of food sources. The angle …

dance language / navigation**: When a forager bee finds a good flower source, it performs the waggle dance to …

waggle dance / navigation**: Bees use the waggle dance to communicate the location of food sources. The angle of the dance relative to the sun tells other bees the direction and distance. This is like an index pointer - you don't go to the data directly, you f…

dance language / navigation**: When a forager bee finds a good flower source, it performs the waggle dance to communicate location to other bees. This is like an index that points to data. But here's a concrete detail: the dance encodes distance and direction …

ach time. They build a mental map - an index - of where flowers are. This maps to query plans and how the opti…

very time. They build a mental map. This is like a cached index vs. a cold index that needs to be rebuilt.

5.…

ach time. They build a mental map - an index - of where flowers are. This maps to query plans and how the optimizer uses indexes. The forager's mental map is the execution plan; the flowers are the data; the index is the cached knowledge of where things are.

…

very time. They build a mental map. This is like a cached index vs. a cold index that needs to be rebuilt.

5. **The queen's pheromone (queen mandibular pheromone)**: This is a single chemical signal that the entire colony references. Every bee "looks up" the …

Full results, leg by leg

Every raw file, one click each, opens in a new tab.

The Table

The Boilerplate

The Essay

Experiment 1, repeated: the baseline run twice

Looking at the Experiment 1 matrix, the three MTP depths matched each other on the Boilerplate and the lone odd one out was MTP off, the plain model. I could not tell from one run whether that means anything, so I reran the MTP-off leg with the identical configuration (thinking on, medium effort, temperature 1.0, top_p 0.95, top_k 20, min_p 0, seed 42, fresh server launch) on a different day and compared it against run 1 and against the MTP depths. The question is simple: does the same setting reproduce itself under sampling, or does it drift the same way the MTP legs did?

Run 2 started 2026-08-25 00:40 local time; every row below carries its own completion stamp.

The Table

The Boilerplate

The Essay

Did run 2 reproduce run 1?

What that means. With seed 42 pinned, the sampled run reproduces itself exactly, thinking and all, on a different day and a fresh server launch. So the reason MTP off stood alone in the Experiment 1 matrix is not that the plain model is noisier or that the first run was a fluke. It is that MTP off decodes one token per step on one kernel path, and every MTP depth verifies a batch of guesses on another; the two paths agree to many decimal places, and where a near-tie token sits between them the seeded lottery lands differently. Same seed, same model, one variable changed, and that variable is the one this whole page is about. Which leg is "right" is not a meaningful question: both are valid samples of the same model, and Experiment 2 shows that with the dice removed the two paths agree byte for byte on structured work.

Legs joined by = produced byte-identical files (first 8 hex digits of the SHA-256 shown). They can be recomputed from the audit links.

Where run 1 and run 2 differ, if anywhere

The Table

✅ identical (answer + thinking) off, run 1 vs off, run 2

The Boilerplate

✅ identical (answer + thinking) off, run 1 vs off, run 2

The Essay

✅ identical (answer + thinking) off, run 1 vs off, run 2

Full results

The Table

The Boilerplate

The Essay

Experiment 2: thinking OFF, greedy, the clean room

Same three prompts, same four settings, but temperature 0 and no thinking: the configuration where outputs should match if MTP truly only changes speed. Verdict up front: byte-identical at every depth on every structured prompt; the Essay still flipped at depths 3-4, the one measured residual (explained below).

Experiment 2's settings, explained parameter by parameter

This experiment deliberately changes two things from the recommended configuration. Here is every knob, what it does, and why it is set the way it is. Worth understanding once, it applies to every local model.

The Table 60 rows of formula-driven inventory data, every cell checkable

Generate a markdown table for a warehouse inventory with EXACTLY 60 product rows plus a final TOTAL row.

Columns: Row | SKU | Product | Unit Price | Stock | Line Value

Rules, follow them exactly:
- Row numbers run 1 through 60.
- SKU for row N is "WH-" followed by N padded to 4 digits (row 7 = WH-0007).
- Product name for row N is "Widget Model N".
- Unit Price for row N is 10.00 + (N * 0.25), shown with 2 decimals.
- Stock for row N is 500 - (3 * N).
- Line Value is Unit Price * Stock, shown with 2 decimals.
- The final TOTAL row shows the sum of the Stock column and the sum of the Line Value column.

Do all arithmetic silently. Your answer must be ONLY the finished markdown table: no calculation steps, no explanations, no text before or after it.

Legs of this test completed 2026-08-23 16:25 → 2026-08-23 17:27 (local time; each row above carries its own stamp).

The Boilerplate a five-resource FastAPI back end, repetitive by design

Write a complete, single-file FastAPI application implementing a bookstore back end with five resources: books, authors, publishers, genres, and customers.

Requirements:
- Each resource gets the identical five endpoints: list all, get by id, create, update, delete.
- Each resource gets its own Pydantic model with at least four typed fields.
- Storage is a plain in-memory dict per resource; ids are integers assigned from a counter.
- Every function has a type-hinted signature and a one-line docstring.
- No authentication, no database, no comments about what you might do differently.

Output only the Python code in a single fenced block, no commentary before or after.

Legs of this test completed 2026-08-23 16:35 → 2026-08-23 17:31 (local time; each row above carries its own stamp).

The Essay 800 words of unpredictable prose, the draft head's worst case

Write a reflective essay of roughly 800 words on what beekeeping can teach a software engineer about database indexing. Draw at least three specific parallels, ground each one in a concrete detail of how bees or hives actually work, and end with one practical lesson a working engineer could apply this week. Avoid bullet points; write flowing prose.

Legs of this test completed 2026-08-23 16:40 → 2026-08-23 17:34 (local time; each row above carries its own stamp).

Did it change the output?

Three views of the same question. First the verdict matrix: every pairing, thinking and answer judged separately. Then who-matched-whom: the legs grouped by file hash, so the pattern is visible at a glance. Then the highlights.

✅ every byte matched · 🟡 same words, only formatting differs · ❌ the answers genuinely differ. Left columns: each depth against MTP off. Right columns: each depth against the previous depth.

Legs joined by = produced byte-identical files (first 8 hex digits of the SHA-256 shown). They can be recomputed from the audit links.

Where exactly they split

Skim this. One card per distinct divergence: the last stretch both legs shared, then what each wrote next. Depths that wrote the same continuation are merged into one card. Identical pairs are stated in one line.

The Table

✅ identical MTP off vs depth 2  ·  ✅ identical MTP off vs depth 3  ·  ✅ identical MTP off vs depth 4

The Boilerplate

✅ identical MTP off vs depth 2  ·  ✅ identical MTP off vs depth 3  ·  ✅ identical MTP off vs depth 4

The Essay

✅ identical MTP off vs depth 2

watching the bees navigate their complex social and physical environment, I realized that the principles gover…

I realized that the bees have been solving the problem of efficient retrieval long before we invented SQL. The…

watching the bees navigate their complex social and physical environment, I realized that the principles governing their efficiency are not so different from those governing our databases. The hive is a living lesson in how structure enables scale, offering pr…

I realized that the bees have been solving the problem of efficient retrieval long before we invented SQL. They do not just store data; they curate it. In their waxen architecture, there is a profound lesson on how to organize information so that access is not…

watching the bees navigate their complex social and physical environment, I realized that the principles gover…

I realized that the bees have been solving the problem of efficient retrieval long before we invented SQL. The…

watching the bees navigate their complex social and physical environment, I realized that the principles governing their efficiency are not so different from those governing our databases. The hive is a living lesson in how structure enables scale, offering pr…

I realized that the bees have been solving the problem of efficient retrieval long before we invented SQL. They do not just store data; they curate it. In their waxen architecture, there is a profound lesson on how to organize information so that access is not…

Full results, leg by leg

Every raw file, one click each, opens in a new tab.

The Table

The Boilerplate

The Essay

Direct comparison: thinking vs no thinking

Two things to notice. The baseline is the same machine-limit either way (~4.2 tok/s at BF16: every token reads all 55 GB). And acceptance, the engine of the speedup, depends on how predictable the TEXT is, thinking transcripts included.

Bonus experiment: the marathon Experiment 1 side: Qwen's factory-default effort (xhigh), a ~12,000-token thinking workload

Generate a markdown table for a warehouse inventory with EXACTLY 60 product rows plus a final TOTAL row.

Columns: Row | SKU | Product | Unit Price | Stock | Line Value

Rules, follow them exactly:
- Row numbers run 1 through 60.
- SKU for row N is "WH-" followed by N padded to 4 digits (row 7 = WH-0007).
- Product name for row N is "Widget Model N".
- Unit Price for row N is 10.00 + (N * 0.25), shown with 2 decimals.
- Stock for row N is 500 - (3 * N).
- Line Value is Unit Price * Stock, shown with 2 decimals.
- The final TOTAL row shows the sum of the Stock column and the sum of the Line Value column.

Output only the table, no commentary before or after.

A 47-minute generation became a 20-minute one, with acceptance holding in the 80s across ~12,000 tokens. MTP does not fade over long runs. Kept apart from the main matrix because the workload differs by design.

Did it change the output?

✅ every byte matched · 🟢 the final ANSWER matched even though the thinking transcript took a different path (different reasoning, same destination) · 🟠 thinking matched but the answer forked (the split happened while writing the answer) · 🟡 same words, only formatting/casing differs · ❌ the answers genuinely differ (equally valid, see the side-by-side splits below). Left columns: each depth against MTP off. Right columns: each depth against the previous depth, which shows whether the MTP legs agree with EACH OTHER.

The side-by-side splits

ment maybe use ---: for numeric? Could use left for text. Output only table. Need no commentary. Ensure exactl…

 maybe right numbers. Need exactly 60 product rows plus final TOTAL. No commentary. Could use:
| Row | SKU | P…

ment maybe use ---: for numeric? Could use left for text. Output only table. Need no commentary. Ensure exactly 60 product rows plus total. Need final TOTAL row. How to label? Maybe:
| 61 |  | TOTAL |  | 24510 | 418492.50 |
But row numbers run 1 through 60. If…

 maybe right numbers. Need exactly 60 product rows plus final TOTAL. No commentary. Could use:
| Row | SKU | Product | Unit Price | Stock | Line Value |
|---:|:---|:---|---:|---:|---:|
| 1 | WH-0001 | Widget Model 1 | 10.25 | 497 | 5094.25 |
...
|  |  | TOTAL …

, 500-180=320.
Line: P*S.
Need format numbers with 2 decimals. Stock integer maybe no decimals? Column Stock l…

 down to 500-180=320.
Line = P*S. Need two decimals. Since price quarters and stock integer, line can be .00,.…

, 500-180=320.
Line: P*S.
Need format numbers with 2 decimals. Stock integer maybe no decimals? Column Stock likely integer. Line value 2 decimals. Unit price 2 decimals.

Need produce 60 rows. Need avoid arithmetic mistakes. Let's compute systematically.

We …

 down to 500-180=320.
Line = P*S. Need two decimals. Since price quarters and stock integer, line can be .00,.25,.50,.75? Stock integer, price multiple .25 => line multiple .25. two decimals.

Let's compute systematically. Could create formula: L_N = 5000 +95N…

Full results, leg by leg

The marathon

Why outputs can differ at all (and why that is expected)

The mechanism, plainly: MTP verification checks several guessed tokens in one batch, normal generation handles one at a time, and GPUs use different math kernels for different batch sizes. The two paths agree to many decimal places, not perfectly. Run with temperature (Experiment 1) and sooner or later one near-tie between two words falls the other way; from that word on, both runs are equally valid paths of the same model. That is exactly what Experiment 1's side-by-side splits show, including the cleanest pattern in the data: depths 2, 3 and 4 share the batched path, so on high-acceptance prompts they landed byte-identical to each other and only differ from "off". Remove the dice (Experiment 2, greedy), and the flipping almost entirely stops: every structured prompt came back byte-identical at every depth. The one residual: the Essay at depths 3 and 4 still flipped, even at temperature 0, because creative prose is full of near-ties so close that the kernel-level numeric difference alone can swap the top word, which is precisely the temp-0 drift a user reported in the maintainer thread. When a split lands inside the thinking transcript, the answer that follows inherits the new path; and sometimes, as the 🟢 highlights show, the diverged thinking still lands on the exact same answer. Nothing in our setup caused any of this: no harness, seed pinned, samplers identical, flags on record. It is the documented reality of speculative decoding on GPUs, not an error, and not something to fix.

Checked, not assumed: the repeat run of the MTP-off leg (same seed, same samplers, a different day) came back byte-identical to the first run on all three prompts, thinking transcript included. So under this protocol the sampler is reproducible, and the only thing that moved any output was switching the decoding path. That is the strongest version of the statement this page can make: the off-vs-MTP fork is systematic, not noise.

Memory cost

Unified-memory rig: CPU and GPU share one 128 GB pool; this is total RAM in use at the highest point, OS included. MTP's draft cache costs roughly 0.5-1.5 GB depending on depth.

What is MTP?

A language model normally writes one word-piece (token) at a time: predict, commit, repeat. Every single token requires reading the entire 55 GB model from memory once. That memory trip is the speed limit.

MTP gives the model a scribbling assistant. Qwen 3.8 ships with a tiny extra "head" trained to guess the next few tokens ahead. At generation time the assistant scribbles 2-4 tokens cheaply, and the full model checks all of them in a single one of those expensive memory trips. Guesses it agrees with are kept: the trip got you several tokens instead of one. The first wrong guess and everything after it is thrown away, and the model continues from there exactly as it would have anyway.

That is why the quality cannot get worse: the big model still approves every single token. The assistant never gets to publish a word the writer did not sign off on. Predictable text (tables, boilerplate) = constant right guesses = you fly. Creative prose = more misses = smaller speedup, which is exactly what the acceptance column shows.

Why the GPU has room for this at all: for every token, the whole model has to be read from memory once, and the arithmetic finishes long before that read does. The compute units mostly wait. Speculative decoding, and MTP as its built-in form, spends that idle compute on checking several drafted tokens per memory trip. That framing comes from Devsplainers' MTP explainer, which also quotes community numbers on Qwen 3.6 27B that match the shape measured here: about 3x on Python at ~90% acceptance, 2 to 2.5x on general chat, closer to 1.8x on creative writing. It also notes a theoretical ceiling: the speedup grows only logarithmically with draft length, so 3 to 5 draft tokens is where the trade-off usually lands, which is consistent with depth 4 beating depth 2 by a modest ~25% on this rig rather than doubling it.

Where MTP came from (a short lineage)

Worth checking if you run llama.cpp: the flag

The MTP head ships inside the GGUF file you already downloaded, at every quant level. But llama.cpp keeps it OFF unless you pass --spec-type draft-mtp --spec-draft-n-max N (support landed in PR #22673, July 2026). Unsloth's launcher passes that flag for you, at depth 2, and does not print it. So a plain llama.cpp launch may be running without it, and an Unsloth launch (including the earlier benchmark runs on this channel) may have had it on without you knowing. Ollama has MTP in a release candidate, and vLLM and TensorRT-LLM support it in production, per the Devsplainers explainer linked below; check your own launcher's flags and log. Toggle it yourself with unsloth run --speculative-type off or sweep depths with --spec-draft-n-max 2..4. On this rig, depth 4 beat the silent default by another ~25%.

When MTP may not help

Not measured here, but worth knowing before you flip the flag. Three cases from the Devsplainers explainer above: mixture-of-experts models, where verifying drafted tokens can route to different experts and load different weight slices for each one (they cite a Qwen 3.6 MoE benchmark on a single RTX 3090 where every speculative configuration was slower than baseline despite high acceptance); high concurrency, since at batch sizes above roughly 16 to 32 the GPU is already compute-bound and there is no idle capacity to spend, so MTP is a latency trick rather than a throughput trick; and small models in the 1 to 3 billion parameter range that already run at 100+ tokens per second, where the drafting overhead becomes a noticeable slice of every step. This page is one dense 27B model at batch size 1, the case MTP was built for.

Same hardware, other people's numbers

kyuz0's Strix Halo MTP benchmarks (same Ryzen AI MAX 395+ / 128 GB platform as mine) measured Qwen 3.6 27B at 1.81x (Q4, depth 3) and 2.44x (Q8). This page adds Qwen 3.8 at BF16 to that picture, on the same silicon. My acceptance rates and multipliers came out higher; I attribute that to prompt design (the Table and Boilerplate prompts are built to be predictable) rather than to anything about the hardware or the build, which is itself a reason to lean on established benchmark suites instead of home-made prompts. More on that in the conclusion.

Validation checklist (everything we checked, and how)

Conclusion

This applies to my machine, my model, my prompts and my two protocols. It is shared for people who are curious what I ran into while researching the topic, structured the way I wanted to experience it.

What the two experiments showed. In Experiment 2 I controlled every variable I could (thinking off, greedy decoding, fixed seed) and got byte-identical output at every MTP depth on the structured prompts, with the essay's near-ties as the one measured residual. In Experiment 1 I let the dice back in (thinking on, temperature 1.0), and the outputs forked between MTP off and MTP on, with the side-by-side splits showing where. Then I reran the MTP-off leg with the same seed and it reproduced itself byte for byte on all three prompts, so the fork is not run-to-run noise: it is the speculative path itself nudging a near-tie token the other way. Both results are what the theory predicts; the point of running them was to see it on my own hardware rather than take it on faith.

What I would do with it. Enable it. For Qwen 3.8 on this rig it takes BF16 from ~4.2 tok/s to ~12.7 tok/s at depth 4 on predictable output, holds real gains on prose, does not fade over a 12,000-token marathon, and costs about a gigabyte of memory. For a model like Qwen 3.8: check the flag in your llama.cpp or launcher config, because the default may not be what you think, and have it on at depth 2 for every kind of task. Depths 3 and 4 are something to test for your own use case rather than a blanket recommendation: on this build they added more speed on coding and structured output and less on creative writing, and whether that trade-off is worth it depends on your hardware and memory (a discrete GPU, a Strix Halo box like mine, a GB10, all different). I can only speak for this specific build and these findings.

Where I am honest about the limits. My acceptance rates beat the published Strix Halo numbers, and the main reason is that my prompts were written to be predictable. That is prompt design, not a hardware result, and it is a skill I am still building. The better move, which I intend to work toward, is to stop reinventing the wheel: run the established benchmark suites, contribute to them where they fall short, and compare against the same prompts other people use, so the numbers are comparable instead of merely mine.

Full reasoning and the story of finding all this: in the video.

Credit where it's due

The explainer that made this click for me: Donato Capitella's MTP video, measured on this same hardware family. My video builds on his foundation with my own rig's numbers; go watch his for the step-by-step mechanics.

The rig all of this ran on

GMKtec EVO-X2 mini PC: AMD Ryzen AI Max+ 395, 128 GB unified memory (CPU and GPU share one pool, which is why a 55 GB model fits at all). Linux (CachyOS), models served by Unsloth's launcher on top of llama.cpp.

My exact build: GMKtec EVO-X2 (Ryzen AI Max+ 395, 128GB)
The newer tower model: GMKtec EVO-X3 (tower cooling + OCuLink)

Some links above are affiliate links. If you buy through them I may earn a small commission at no extra cost to you. It helps support the channel.
As an Amazon Associate I earn from qualifying purchases.

All sources

• Donato Capitella: MTP, 2x faster token generation on AMD Strix Halo (the explainer this builds on)
• Devsplainers: Multi-Token Prediction, Why Your GPU Runs LLMs 3x Faster (memory-bandwidth framing, the logarithmic ceiling, and the MoE / concurrency / small-model caveats)
• Leviathan et al. 2023 (Google): Fast Inference from Transformers via Speculative Decoding
• Gloeckle et al. 2024 (Meta): Better & Faster LLMs via Multi-token Prediction
• DeepSeek-V3 Technical Report · Shirley Li's MTP walkthrough
• llama.cpp PR #22673 (MTP support) · issue #23302 · issue #23335 (the maintainer's answer)
• sudoingX/qwen38-mtp: per-GPU paired benchmarks + probe tool
• kyuz0: Strix Halo MTP benchmarks (same platform)
• Qwen 3.8-27B model card · the GGUF files · Unsloth's Qwen 3.8 docs

Made by Jose (hijoseromero.com) · every number above was measured on the machine described, logs retained. Figuring out tech, together.