Run DeepSeek V4.1 Flash locally on one 32 GB GPU
DeepSeek V4.1 Flash is a mixture-of-experts model with 763 billion parameters. As a 2-bit GGUF it is 232 GiB, seven times the memory of a 32 GB card. FLE Link runs it anyway: the FLE engine keeps the experts in RAM, puts the busiest ones on the GPU and lets a small draft model (DSpark) propose tokens that the big model checks.
flelink run deepseek-v41-ista-dsparkflelink run rin-deepseek-v41-dsparkWhat you need
One NVIDIA card with 32 GB, 128 GB of RAM and about 260 GB on a fast NVMe. Of the 128 GB the model takes 120 GB, pinned so it cannot be swapped out; 8 GB stay with the system. FLE Link installs the FLE engine (v1.2.5 or newer) by itself.
Measured speed
| Recipe | Decode | Measured on |
|---|---|---|
| DeepSeek V4.1 Flash + DSpark763B mixture of experts, 232 GiB | 49.9 tok/s | 32 GB card, 128 GB RAM (120 GB for the model, 8 GB left to the system), through FLE Link, median of three (5 Nov 2026); 37.2 without the draft |
| Rin (DeepSeek V4.1 Flash, split)763B, hot half on the GPU | 53.4 tok/s | 32 GB card, 128 GB RAM, FLE v1.2.6; lossy, see the model card |
Does DeepSeek V4.1 Flash + DSpark fit my card?
| Graphics | 32 GB RAM | 64 GB RAM | 128 GB RAM |
|---|---|---|---|
| 12 GB NVIDIA card | too big | too big | too big |
| 16 GB NVIDIA card | too big | too big | too big |
| 24 GB NVIDIA card | too big | too big | too big |
| 32 GB NVIDIA card | too big | too big | fits |
| Apple Silicon Mac | no engine | no engine | no engine |
| No GPU | no engine | no engine | no engine |
"no engine": this recipe needs the FLE engine, which has an NVIDIA CUDA build only. File size 237 GiB including the draft model.
Does Rin (DeepSeek V4.1 Flash, split) fit my card?
| Graphics | 32 GB RAM | 64 GB RAM | 128 GB RAM |
|---|---|---|---|
| 12 GB NVIDIA card | too big | too big | too big |
| 16 GB NVIDIA card | too big | too big | too big |
| 24 GB NVIDIA card | too big | too big | too big |
| 32 GB NVIDIA card | too big | too big | fits |
| Apple Silicon Mac | no engine | no engine | no engine |
| No GPU | no engine | no engine | no engine |
"no engine": this recipe needs the FLE engine, which has an NVIDIA CUDA build only. File size 226 GiB including the draft model.
The verdicts are the rule FLE Link applies before it downloads anything (the rules in full). A speed is shown only for the hardware it was measured on. A size nobody measured yet? flelink bench --task pagoda --share posts yours to the benchmarks board.
Structured output and tool calls
A 2-bit quant that agrees with the original on 83.5 % of next tokens: do strict formats break? Measured on 11 Nov 2026 with the same quant (ISTA IQ2_XXS), thinking on, and for the JSON rows without a grammar or JSON mode:
| Task | Result | Details |
|---|---|---|
| JSON to a given schema | 26/26 valid | nested objects, enums, escaped strings; no markdown fences, no extra text |
| Long strict output | 20 and 40 rows valid | no repeated rows; the 60-row case hit the 4000-token cap at row 51, clean up to there |
| Single tool calls, 8 tools | 16/16 | right tool and valid arguments |
| Short agent loops | 5/5 solved | 2 to 3 tool steps with simulated tools; 11 calls, none invalid, none repeated |
A small sample: short loops, English only, and it was not checked whether the server constrains the tool-call syntax itself. Agent runs over hours are not covered; that is where a 2-bit quant would be expected to show it first.
Good to know
- The recipe
deepseek-v41-ista-dsparkuses ISTA DASLab's GSQ-RCO quant (IQ2_XXS) with the DSpark draft model. The draft does not change the answer: the model checks every drafted token. - Rin (
rin-deepseek-v41-dspark) is the same model split into a hot half that stays on the GPU and a cold half served from RAM. It is a little faster and a little worse than the quant it is made from; the model card has the numbers. - The 2-bit quant itself is lossy: its next token agrees with the original weights 83.5 to 85.2 % of the time.
Sources
Install FLE Link
curl -fsSL https://flelink.run/install.sh | shWindows: irm https://flelink.run/install.ps1 | iex. Open source, MIT. No account, no telemetry. What FLE Link does →