Run Qwen3.8-Flash-Next locally on one GPU
Qwen3.8-Flash-Next is a mixture-of-experts model with 125 billion parameters. The FLE quant (FLE-IQ3_M) is an 88.9 GB file, far more than a graphics card holds. The FLE engine keeps the experts in RAM and the busiest ones on the GPU, so the model runs from a 16 GB card and 64 GB of RAM.
flelink run fle-iq3-mWhat you need
One NVIDIA card from 16 GB and 64 GB of RAM; the top speed was measured with 32 GB and 128 GB of RAM.
Measured speed
| Recipe | Decode | Measured on |
|---|---|---|
| Qwen3.8-Flash-Next125B mixture of experts, 88.9 GB | 91 tok/s | 32 GB card, 128 GB RAM, with the engine's MTP draft head started by hand; flelink run starts it without the head so far |
| Qwen3.8-Flash-Next125B mixture of experts, 88.9 GB | about 15 tok/s | 16 GB card, 64 GB RAM (8 Nov 2026) |
Does Qwen3.8-Flash-Next fit my card?
| Graphics | 32 GB RAM | 64 GB RAM | 128 GB RAM |
|---|---|---|---|
| 12 GB NVIDIA card | too big | tight | tight |
| 16 GB NVIDIA card | too big | fits | fits |
| 24 GB NVIDIA card | too big | fits | fits |
| 32 GB NVIDIA card | too big | fits | fits |
| Apple Silicon Mac | no engine | no engine | no engine |
| No GPU | no engine | no engine | no engine |
"no engine": this recipe needs the FLE engine, which has an NVIDIA CUDA build only. File size 83 GiB including the draft model.
The verdicts are the rule FLE Link applies before it downloads anything (the rules in full). A speed is shown only for the hardware it was measured on. A size nobody measured yet? flelink bench --task pagoda --share posts yours to the benchmarks board.
Good to know
- On the same file and the same 32 GB card the FLE engine decoded at 91 tokens/s, Strata at 74 and llama.cpp at 36 (median over six tasks, 6 Oct 2026).
- The 91 tokens/s were measured with the engine's MTP draft head started by hand;
flelink runstarts the model without the head so far.
Sources
Install FLE Link
curl -fsSL https://flelink.run/install.sh | shWindows: irm https://flelink.run/install.ps1 | iex. Open source, MIT. No account, no telemetry. What FLE Link does →