Jose Romero
← All letters
Prefer email? Read this on Substack

Qwen 3.8-27B won every test. I'm not switching

Qwen 3.6-35B has been my local AI of choice for months: quick, reliable, no complaints, running everything from day-to-day questions to agent work, all private, all on my own machine. Then Qwen 3.8-27B arrived with a lot of hype. So instead of trusting anyone's benchmarks, I built my own exam and made them fight.

See it for yourself

Everything in this video is checkable. Open the full results page → every prompt, all 12 playable outputs (both Qwens plus the Opus wildcard), and the numbers.

The setup, because fair matters

Both models ran at full BF16 precision, no quantization, on my 128 GB unified-memory mini PC. Same four one-shot tests each: build a Tetris-style game, simulate an erupting volcano in the browser, turn a deliberately messy CSV into a correct dashboard, and draw a scene in pure SVG code, blind. One prompt, one shot, no retries, thinking mode off for both (identical official settings), and server-side tools disabled so nobody could quietly run Python behind my back. Claude Opus 4.6 rode along as a cloud wildcard; its exam cost about $7 in API tokens while the locals ran on electricity.

The result: a clean sweep

Qwen 3.8 won all four tests. Its Tetris clone had a title screen, levels, scoring, and particle effects. Its volcano sim had sliders, an FPS counter, and an eruption trigger (even if some lava rolled uphill). Its spreadsheet dashboard got every number exactly right, down to the $32,083.00 grand total I had verified by script. Its SVG dog on a forklift was, honestly, the better dog.

So why am I not switching?

Speed. Qwen 3.8 generated at about 8.8 tokens per second on my rig; Qwen 3.6 did 27. The whole exam took 3.8 nearly an hour, while 3.6 finished in 14 minutes. The reason is physics, not configuration: a dense 55 GB model has to read essentially all 55 GB of weights for every single token, and at ~256 GB/s of memory bandwidth that caps out around 5-9 tokens per second no matter how much RAM you have. RAM makes a model FIT. Bandwidth makes it FAST. Qwen 3.6 is a mixture-of-experts model that only activates ~3B parameters per token, which is the entire 3x speed difference, from a bigger file.

The weirdest find of the night

It came during editing. The drawing test asked each model to sign the artwork with its own model name. Qwen 3.8, running entirely on my machine with no cloud involved, signed its drawing "Claude". That strongly suggests Claude-generated text somewhere in its training data, which is simply how open models get trained in 2026. Ask a model its name sideways and sometimes you get its teacher's.

Next: the quant ladder

Quantization attacks exactly the bandwidth problem. The 4-bit version of Qwen 3.8 reads ~14 GB per token instead of 55. The same exam on the 8, 4, and 2-bit versions is the next letter, and if you only ever download the 4-bit like most people, that one is for you.

My rig

Gear in this one

Some links on this page are affiliate links. If you buy through them I may earn a small commission at no extra cost to you. It helps support the work. As an Amazon Associate I earn from qualifying purchases.

Sources

I am documenting a life of interest and curiosity, tech, money, health, and family included. New video daily.

Get the letters

Every post goes out free on Substack. What I build, break, and fix, written up in plain English. No spam, unsubscribe anytime.

Subscribe on Substack