FLE Link

FLE Link / Models

Run DeepSeek V4.1 Flash locally on one 32 GB GPU

DeepSeek V4.1 Flash is a mixture-of-experts model with 763 billion parameters. As a 2-bit GGUF it is 232 GiB, seven times the memory of a 32 GB card. FLE Link runs it anyway: the FLE engine keeps the experts in RAM, puts the busiest ones on the GPU and lets a small draft model (DSpark) propose tokens that the big model checks.

flelink run deepseek-v41-ista-dspark
flelink run rin-deepseek-v41-dspark

What you need

One NVIDIA card with 32 GB, 128 GB of RAM and about 260 GB on a fast NVMe. Of the 128 GB the model takes 120 GB, pinned so it cannot be swapped out; 8 GB stay with the system. FLE Link installs the FLE engine (v1.2.5 or newer) by itself.

Measured speed

RecipeDecodeMeasured on
DeepSeek V4.1 Flash + DSpark763B mixture of experts, 232 GiB49.9 tok/s32 GB card, 128 GB RAM (120 GB for the model, 8 GB left to the system), through FLE Link, median of three (5 Nov 2026); 37.2 without the draft
Rin (DeepSeek V4.1 Flash, split)763B, hot half on the GPU53.4 tok/s32 GB card, 128 GB RAM, FLE v1.2.6; lossy, see the model card

Does DeepSeek V4.1 Flash + DSpark fit my card?

Graphics32 GB RAM64 GB RAM128 GB RAM
12 GB NVIDIA cardtoo bigtoo bigtoo big
16 GB NVIDIA cardtoo bigtoo bigtoo big
24 GB NVIDIA cardtoo bigtoo bigtoo big
32 GB NVIDIA cardtoo bigtoo bigfits
Apple Silicon Macno engineno engineno engine
No GPUno engineno engineno engine

"no engine": this recipe needs the FLE engine, which has an NVIDIA CUDA build only. File size 237 GiB including the draft model.

Does Rin (DeepSeek V4.1 Flash, split) fit my card?

Graphics32 GB RAM64 GB RAM128 GB RAM
12 GB NVIDIA cardtoo bigtoo bigtoo big
16 GB NVIDIA cardtoo bigtoo bigtoo big
24 GB NVIDIA cardtoo bigtoo bigtoo big
32 GB NVIDIA cardtoo bigtoo bigfits
Apple Silicon Macno engineno engineno engine
No GPUno engineno engineno engine

"no engine": this recipe needs the FLE engine, which has an NVIDIA CUDA build only. File size 226 GiB including the draft model.

The verdicts are the rule FLE Link applies before it downloads anything (the rules in full). A speed is shown only for the hardware it was measured on. A size nobody measured yet? flelink bench --task pagoda --share posts yours to the benchmarks board.

Structured output and tool calls

A 2-bit quant that agrees with the original on 83.5 % of next tokens: do strict formats break? Measured on 11 Nov 2026 with the same quant (ISTA IQ2_XXS), thinking on, and for the JSON rows without a grammar or JSON mode:

TaskResultDetails
JSON to a given schema26/26 validnested objects, enums, escaped strings; no markdown fences, no extra text
Long strict output20 and 40 rows validno repeated rows; the 60-row case hit the 4000-token cap at row 51, clean up to there
Single tool calls, 8 tools16/16right tool and valid arguments
Short agent loops5/5 solved2 to 3 tool steps with simulated tools; 11 calls, none invalid, none repeated

A small sample: short loops, English only, and it was not checked whether the server constrains the tool-call syntax itself. Agent runs over hours are not covered; that is where a 2-bit quant would be expected to show it first.

Good to know

Sources

Install FLE Link

curl -fsSL https://flelink.run/install.sh | sh

Windows: irm https://flelink.run/install.ps1 | iex. Open source, MIT. No account, no telemetry. What FLE Link does →