Running Qwen 3.8 Flash-Next at xhigh on 3 harnesses

Qwen 3.8 Flash-Next is good in chat on my box. I give it prompts, it does quick file management, and it answers at 20 to 24 tokens a second. So I started trusting it with more, and plugged it into the harnesses I leave running overnight: the agent jobs that build my slides, work on channel stuff and do personal admin while I sleep. That is where it got slower, and where I started seeing more errors than I did with Qwen 3.6.

My first guess was the harness. So I ran eighteen benchmark jobs to find out: every task in my harness-bench suite, on three harnesses, with the same model and the same server. The short answer is that it is not the harness. It is what happens to a local model as you keep feeding it context.

The setup

The box is the one every benchmark on this channel runs on: a GMKtec EVO-X2 (paid link), Ryzen AI Max+ 395, 128 GB of unified memory, CachyOS. Strix Halo is not fast on memory bandwidth, but I have had good luck with local AI on it.

The model is Unsloth's UD-IQ4_XS build of Qwen 3.8 Flash-Next, 93.7 GB, on stock llama.cpp with Vulkan. Context window 131,072 tokens. Sampling is Qwen's recommended thinking set. Thinking is on extra high: I run medium for anything simple and extra high for the agent work I leave overnight, and every earlier harness run on this channel was medium, so this one is not directly comparable to those.

Six tasks, three harnesses (OpenCode, Pi and DeepSeek Harness), one run each: eighteen runs, a two-hour cap per run. All three harnesses ran their out-of-the-box config for this, even though my daily Pi and DeepSeek Harness setups have extra tools and skills. The tasks are not visual. Each one is a folder and an instruction: fix this failing test suite, write this log analyzer, consolidate these expense exports. After each run the suite takes a diff, checks it against a scoring key, and then a judge model, Qwen 3.8 27B at Q4, reads what the agent touched and grades the process against a rubric. That is AI grading AI, with a rubric to hold it to.

It is not the harness

Seventeen of eighteen runs resolved. Every OpenCode and DeepSeek Harness run scored 100. Pi missed one checkpoint on one task and finished at 95, the only partial score of the night. The runs took anywhere from 4 minutes to 100, and the long ones are where the story is.

Here is generation speed by how much context the model was carrying when it answered:

Under 25k tokens: about 15 tokens a second.
25k to 50k: about 12.
50k to 75k: about 9.2.
Over 75k: about 8.1.

That curve is the same on all three harnesses. The short tasks averaged 7 minutes and ran at about 15 tokens a second. The long ones ran an hour or more, carried up to 97k tokens of context, and slowed down no matter which harness drove them. There are fewer requests at the high end, eighteen past 75k against 232 under 25k, so the top of the curve has the thinnest data. It still lines up with what I felt in the overnight runs.

Where the harnesses do differ is time and turn count. DeepSeek Harness was the slowest overall and degraded the most on the long tasks. OpenCode and Pi were a little better. Out of the box, Pi felt the most efficient to me, and Qwen seems to work a little better with it. That is a sample size of me, so take it with a grain of salt.

Why n=1 matters

This is one run per task per harness. If I wanted to be more accurate I would run each one several times, but some of these take an hour, times eighteen, overnight, on the desktop I also game and work on. It cannot run benchmarks around the clock. So the scores tell you the model can finish these jobs on every harness. They cannot separate a 94% setup from a 92% one.

One more honest limit: pulling the thinking out of each run was easy on OpenCode and hard on Pi and DeepSeek Harness through their CLIs. The page now counts the thinking from each harness's own session files.

What about MTP

MTP was off for all of this, on purpose. Multi-token prediction lets small extra heads guess the next few tokens while the main model checks them, and when the guesses land, generation gets faster. For Flash-Next it is not in stock llama.cpp yet: the pull request is still open as a draft.

Then I read Unsloth's MTP guide, which says 1.3 to 1.7 times faster. It works through their own llama.cpp branch plus a separate sidecar file, and Unsloth Studio sets it up for you: it offered to download the sidecar as soon as I loaded the model, and speculative decoding was already set to auto. In a quick chat it answered at about 21 to 22 tokens a second, which is not much of a difference from what I get without it, at least in my setup. It also left me with about 4 GB of free memory with nothing else running but OBS, which is not usable as a daily setup. If I go the MTP route I will probably drop to a Q3.

The model's own explain-it-like-I'm-five was good: you write one letter at a time, two helpers guess the next ones, and when you check and they were right, you just wrote three turns in one. The part it skipped is the cost. When the guesses are wrong often enough, you pay for them, and the helper model takes memory and bandwidth running side by side.

Why I ran this now

Donato Capitella's video from September 11 covers exactly what I run: Qwen 3.8 Flash-Next on Strix Halo, with Halogen, N-grams and benchmarks. If you run a Strix Halo, follow his channel. His testing is the reason I found Halogen, a server built only for this model on this chip, and its claims are a big jump in both generation and prefill.

Most people will not use a model like this for chat or one-shot tests. They will put it in a harness. So I wanted verified numbers for my current setup first, and then the same eighteen jobs on Halogen. Around 15 to 20 tokens a second is acceptable, in quotes. If Halogen gets it into the 30s and 40s, it becomes my favorite setup, at least until MTP lands in llama.cpp and I run this again.

Which harness do you run local models through, and how much context do you actually hit in a session?

Gear in this one

Affiliate links may earn me a commission at no extra cost to you. As an Amazon Associate I earn from qualifying purchases.