Qwen 3.8 Flash-Next on Halogen: 72 coding runs later

My previous Qwen 3.8 Flash-Next test had a frustrating pattern: the longer the agent worked, the slower generation became. The model could do useful work on my 128 GB Strix Halo box, but waiting at eight or ten tokens per second made it hard to use interactively.

After watching Donato Capitella's video about Halogen, I tried its dedicated inference server with the same model family. This time I ran 72 coding jobs across OpenCode, Pi and DeepSeek Harness, or dsh. The improvement was enough that I want to keep using it for my own work.

The important part is what changed, what passed, and what these numbers can actually tell us.

The headline result

Halogen averaged 38.0 tokens per second in the per-run generation measurements. My earlier llama.cpp configuration averaged 13.9. That is about 2.7 times the generation rate.

For me, moving into the thirty-to-forty-token range changes how this feels. I have been trying it with channel work, OBS-related tasks and other small jobs. It feels more responsive. That is my practical experience, separate from the benchmark measurements.

The speed also held up as the working context grew. Below 25,000 context tokens, Halogen averaged 39.5 tokens per second against 17.8 for the earlier configuration. At 75,000 and above, the averages were 37.8 and 8.2.

Those are groups of requests from agent sessions, not identical prompts replayed through two engines. The deepest group contains 33 requests for Halogen and 18 for the earlier setup. It is encouraging evidence from this workload, not a promise about every context length or coding project.

Faster still needs to finish

I used six tasks, three harnesses and four repeats per harness/task pair. Of the 72 jobs, 66 passed every checkpoint.

OpenCode passed 24 of 24. dsh passed 22 of 24. Pi passed 20 of 24. All jobs stopped within their limits, but stopping is not the same as passing every check.

The earlier campaign passed 17 of 18, with only one attempt per pair. I described the lower pass percentage as degradation while talking through the results. That is too strong a conclusion from these two campaigns. The repetition counts and configurations differ, and this is a small task set. It does not establish that Halogen caused a reliability regression.

The detailed reports let you inspect the checkpoints, code changes and available agent output. That is more useful than asking you to take one score on trust. A passing result still means passing this suite, not proving production-ready code.

This was a configuration comparison

The earlier run used an Unsloth UD-IQ4_XS GGUF through llama.cpp with Vulkan and no speculative decoding. The Halogen 0.11.3 run used W4B plus a quality overlay, ROCm and MTP drafting.

Context limits changed from 131,072 to 163,840. The earlier jobs had a 120-minute cap; these jobs had a 60-minute cap. Neither campaign recorded timeouts. The task and prompt hashes, suite and harness versions match, but enough other things changed that I cannot assign the whole speed difference to one feature.

Prefill improved too. The mean of the per-run prefill rates was 557 tokens per second versus 145. Prefill is input processing; generation is producing the next tokens. They are separate measurements, and neither is a direct measure of how quickly a whole project gets finished.

One correction to my quick commentary: available-memory and swap readings do not establish a memory saving or prove where the weights reside. I would need a more controlled memory measurement to make that claim.

Where I go next

I plan to keep running this setup in my own harness workflows. In the video I also looked at EngramHalo.cpp, an open alternative worth following. I have not benchmarked that fork here, so its advertised results are not my measurements.

Halogen's runtime was closed source in the configuration I tested. The performance is encouraging, but a benchmark does not settle the trust decision or guarantee future release plans.

For the next comparison, I want better-matched conditions and more repeated work. For now, the useful result is straightforward: generation stayed much faster on this box, most jobs passed, and I have the individual runs available for anyone who wants to check my interpretation. If you spot something I should measure differently, I want that feedback.

Sources

• My 72-run Halogen results
• My llama.cpp and Halogen comparison
• Halogen Flash Server
• The Halogen checkpoint
• Donato Capitella's video that prompted the test
• EngramHalo.cpp