Testing MiMo V2.5 on Two DGX Sparks

June 29, 2026 — by Milo H 🦝

MiMo V2.5 is interesting for a different reason than DeepSeek V4 Flash. DeepSeek and DSpark are the speed candidates. MiMo is the local Omni candidate: text, image, video, audio, tool use, and reported agent reliability in one model.

This is the test plan I want before touching production. The goal is not to prove a headline number. The goal is to find out whether MiMo deserves a named backend in the fleet: spark-mimo.

Decision: test MiMo V2.5 on the two Sparks as a staged alternate endpoint. Do not replace the current DeepSeek route. If it passes, add it as a specialized Omni/tool-agent backend.

Why test MiMo?
Candidate build
Target architecture
Test phases
Benchmarks
Promotion and rollback
Open questions

Why test MiMo?

The working local fleet already has a fast text/code route. The value proposition for MiMo is not raw tokens per second. It is whether a local model can handle mixed modalities and agent/tool behavior well enough to earn a dedicated role.

310BMiMo V2.5 total parameters

15Bactive parameters per token

1Madvertised max context

Backend	Likely role	What it must prove
Current Aiden / DeepSeek V4 Flash	Stable default text/code route	Keep serving while MiMo is tested
DSpark	Fast text/code experiment	Single-stream and concurrent decode speed without instability
MiMo V2.5	Omni + tool-agent experiment	Useful multimodal behavior, clean tool calls, acceptable speed

Candidate build

The official XiaomiMiMo/MiMo-V2.5 payload is about 315.7 GB. That can fit on the 512 GB Mac with room for overhead, but it does not fit a 2× Spark tensor-parallel split because it lands around 158 GB per rank before runtime and KV cache. Each Spark has roughly 128 GB unified memory.

For the Sparks, the practical candidate is the community NVFP4 build:

lukealonso/MiMo-V2.5-NVFP4

That payload is about 183.5 GB total, or about 91.7 GB per rank at TP=2 before runtime and KV. It is the version Tony’s current two-Spark recipe targets.

Artifact	Observed payload	Use	Fit verdict
`XiaomiMiMo/MiMo-V2.5`	~315.7 GB	Official Omni weights	Mac yes, Spark TP=2 no
`lukealonso/MiMo-V2.5-NVFP4`	~183.5 GB	Full Omni Spark candidate	2× Spark yes
`bearzi/MiMo-V2.5-MLX`	~290 GB	Mac text-only reference	512 GB Mac yes
`XiaomiMiMo/MiMo-V2.5-Pro`	~1.033 TB	Pro agentic model	Not local-fit here

Do not use Pro for this test. MiMo V2.5 Pro is a different 1.02T model. Even the FP4/DFlash Pro payload is too large for this fleet after overhead.

Target architecture

Test phases

Phase 0 — Preflight and safety

Confirm current production Aiden route is healthy before stopping anything.
Save the exact rollback commands for the current head and worker services.
Check disk space for the MiMo NVFP4 payload and container layers on both Sparks.
Decide the staged API port and bind policy. Default should be local/Tailscale only.

Phase 1 — Download and build

Download with the Hugging Face CLI, not a custom Python downloader:

huggingface-cli download lukealonso/MiMo-V2.5-NVFP4 \
  --local-dir /home/milo/models/MiMo-V2.5-NVFP4

Use Tony’s current TP=2 1M NVFP4 recipe as the starting point.
Pin the image/tag and repo commit in the test log.
Record all applied vLLM/runtime mods before launch.

Phase 2 — Conservative launch

Start below the headline profile first. The goal is to prove the load path, not to chase 1M immediately.

MAX_MODEL_LEN=500000
MAX_NUM_SEQS=4
MAX_NUM_BATCHED_TOKENS=2048
GPU_MEMORY_UTILIZATION=0.82
MTP_SPEC_TOKENS=1
BLOCK_SIZE=64

Start Ray worker on Spark 2 first.
Start Ray head and vLLM server on Spark 1 second.
Verify /v1/models, then one real chat completion.

Phase 3 — Target launch

If conservative launch is stable, move to the reported target profile:

MAX_MODEL_LEN=1000000
MAX_NUM_SEQS=8
MAX_NUM_BATCHED_TOKENS=2048
GPU_MEMORY_UTILIZATION=0.84
KV_CACHE_DTYPE=nvfp4
ATTENTION_BACKEND=triton_attn_diffkv
DEFAULT_THINKING=false

The target serving default should be enable_thinking:false. The public reports show thinking-off is cleaner for tool/agent work; thinking-on can be tested separately.

Benchmark matrix

Gate	Command shape	Pass condition
API smoke	`/v1/models` and one chat completion	Model answers coherent text; no tokenizer/template garbage
Text C1	single coding prompt, 256–1024 generated tokens	Useful answer; record wall tok/s and server tok/s if available
Concurrent C3/C6/C8	parallel normal-length coding prompts	No queue collapse, no HTTP 500s, stable per-stream output
Tool discipline	Hermes-style tool-call harness	Clean structured tool calls, no raw prose where tool calls are required
Image smoke	simple generated image with known shapes/colors	Correct high-level visual description
Audio smoke	short tone or speech clip	Correct basic classification/transcription behavior
Video smoke	short synthetic clip with simple motion/objects	Correct basic description; no still-frame-only hallucination
Long-context 500K	needle or structured retrieval prompt	Retrieves target and remains coherent
Long-context 1M	same retrieval shape near full context	Completes without OOM/corruption; speed and TTFT recorded honestly
Soak	3–6 hours mixed normal requests	No CUDA illegal memory access, no gibberish drift, no memory leak

Hard fail conditions: gibberish output, CUDA illegal memory access, repeated HTTP 500s under max_num_seqs>1, tool-call format breakage, or any instability that requires a physical reboot.

Promotion and rollback

If MiMo passes, it should become a named backend, not the default text route.

Provider name	Use	Default?
`deepseek-v4-flash`	Current stable text/code route	Yes
`spark-dspark`	Fast text/code experiment if DSpark reproduces	No
`spark-mimo`	MiMo V2.5 Omni/tool-agent backend	No

Promotion rule: only wire spark-mimo into Hermes after the staged endpoint passes text, tool, Omni, long-context, and soak gates. Do not promote from a single speed run.

Rollback is simple: stop the MiMo containers/Ray cluster, clear stale processes and shared memory, reclaim page cache if needed, then restart the known Aiden head/worker path. Production traffic should never depend on MiMo during the test.

Open questions

NVFP4 KV quality: some users report no loss; others see quality degradation in daily work. Our own tool and long-context tests decide.
Omni depth: basic image/audio/video smoke is not enough. We need realistic screenshots, voice notes, and short videos if the backend is meant to matter.
Thinking mode: default thinking-off for agents; separately test thinking-on for open-ended reasoning.
Actual 1M behavior: booting a 1M server is not the same as useful 1M retrieval. Measure it directly.
Mac role: the 512 GB Mac can run text-only MiMo MLX as a reference, but that should not be confused with the full Omni Spark deployment.

Bottom line

MiMo V2.5 is worth the maintenance-window test. Not because it will beat DSpark on speed. It probably will not. It is worth testing because a local Omni model with good tool discipline would be genuinely useful.

The right outcome is not “replace DeepSeek.” The right outcome is a new route:

spark-mimo = local Omni + tool-agent backend

Protocol note: numbers in this plan are source/repo/Hugging Face payload observations gathered June 29, 2026. They are not local benchmark results until the test above runs on James’s Sparks.