modelrigs

How to run DeepSeek-V4.1 Flash locally

Select your machine for accelerator-specific, revision-pinned commands using DwarfStar.

Model benchmarks
Choose your hardware

Showing 15 compatible configurations at 8K context. Selection changes the instructions, not the canonical URL.

Runtime choices are complementary. Changing runtime selects a different artifact, memory profile and command set; it does not replace the other method.

Recommended backend
Apple Metal
Calculated macOS
Artifact
DS4 V4.1 Q2 (SSD)
Estimated GGUF · antirez/ds4
Memory required
93.2 GB
Calculated of 108.8 GB usable
Fit at 8K context
Fits
throughput unavailable
Verification status: artifact and download instructions checked against the runtime repository on 2026-09-14; command generated from a unit-checked backend template; not executed on physical MacBook Pro M4 Max 128 GB hardware.
Text-only quick start: this recipe starts a text chat. Using the model's vision capability can require a matching multimodal projector and additional llama.cpp flags.
Quick start · DwarfStarNot hardware tested
1. Install DwarfStar
git clone https://github.com/antirez/ds4.git
cd ds4
git checkout 6e4c285a92a1de11863784b748f196b5d4b88a48
make
./download_model.sh ds41f-q2
2. Start a local chat
./ds4 -m gguf/DeepSeek-V4.1-Flash-Q2.gguf --ssd-streaming --ctx 8192

The first run downloads the model into a persistent cache. Large artifacts can take a long time; make sure the machine has enough free disk space before starting.

Run an OpenAI-compatible local API
Start the server
./ds4-server -m gguf/DeepSeek-V4.1-Flash-Q2.gguf --ssd-streaming --ctx 8192
Test it from another terminal
curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"deepseek-v4-1-flash","messages":[{"role":"user","content":"Reply with one short sentence: what hardware are you running on?"}]}'
The server binds to loopback by default. Do not expose the server to your network or the public internet until you have added authentication and appropriate access controls.
Prerequisites for MacBook Pro M4 Max 128 GB
  • Apple Silicon with current Xcode command-line tools.
  • At least 96 GB unified memory for resident execution and approximately 81 GiB of free disk.

DwarfStar backend documentation ↗

Why this recipe
  • The repository and commands are pinned to the same DwarfStar revision as the imported benchmark dataset.
  • The server listens on 127.0.0.1:8000 by default and exposes OpenAI-style, Responses-style and Anthropic-style endpoints.
  • Keep --ssd-streaming enabled on one Spark and start without speculative decoding; DwarfStar sizes the expert cache automatically.
  • This 340.598 GiB artifact cannot run resident on one 128 GB DGX Spark; DwarfStar streams routed experts and the 188.833 GiB Engram tables from a fast local SSD.
  • Automatic cache sizing retained 80.17 GiB of experts in the published single-Spark measurements. Leave it automatic unless you are reproducing a documented explicit-cache test.
  • The published 1K-16K sweep generated about 8.2-9.6 tokens/s without speculative decoding. V4.1 CUDA vision and DSpark are not supported at this revision.
Artifact and assumptions
Repository
antirez/ds4 Estimated
Pinned revision
6e4c285a92a1 Estimated source
Artifact quantization
DS4 V4.1 Q2 (SSD) Estimated
Context in commands ⓘ
8,192 tokens Assumption
Execution mode ⓘ
Fully resident Official spec
Hardware execution
Not tested by ModelRigs Estimated
Operational notes
Memory and context
Start at 8K. Longer prompts grow the KV cache; return to the fit calculation before raising the context substantially.
Persistent downloads
Docker recipes store Hugging Face downloads in the named volume llama-model-cache. Removing that volume also removes the cached weights.
Performance
The page reports measured throughput when available and otherwise shows an estimate range. Runtime build, drivers, prompt length and thermals can all move it.