openbmb/MiniCPM-V-4.7-35B-A3B

TL;DR: MiniCPM-V 4.7 35B-A3B is OpenBMB's first MoE vision-language model (35B total, 3B active, Qwen3.6 backbone + SigLIP2 vision), and the headline is a systems trick rather than the model: Lookahead Expert Scheduling (LES) predicts which experts the next layer will route to and prefetches them, so a model whose expert weights don't fit in phone RAM can still decode at a claimed 13+ tokens/s in ~3 GB on an iPhone 17 Pro Max — without restricting routing. If the claim holds up, this is the pattern that makes big sparse models practical on constrained devices generally.

The core idea: predictive expert prefetching

The on-device MoE problem is well framed in the card: total parameters vastly exceed what fits in memory, and token-level routing means you can't know in advance which experts you'll need. The two standard escapes are both bad — stream experts from flash on demand (correct but stalls decoding) or pin a fixed expert subset per prompt (fast but silently degrades the model by overriding the router).

LES threads the needle: a cross-layer routing predictor guesses which experts layer N+1 will want while layer N computes, prefetches them, then at the actual routing step prioritizes computation on the experts it got right while loading the misses in parallel. Crucially, predictions only affect scheduling, never routing — the model always computes exactly the experts its router selected, so there's no capability loss by construction, only variable latency depending on prediction accuracy. This is combined with fine-grained layer-wise quantization (shrinks the weights being streamed) and a released speculative-decoding draft model ("DSpark") to multiply tokens per expert-load.

The model itself

Architecturally it's the MiniCPM-V recipe with two efficiency levers. First, intra-ViT early compression (from LLaVA-UHD v4): visual tokens are merged inside the vision encoder, cutting compute in later encoder layers rather than just at the projector. Second, runtime-switchable 4x vs 16x visual token compression — you pass downsample_mode per request, so OCR/PDF parsing can run at 4x while video summarization runs at 16x. Video support is reasonably rich: up to 128 main frames, optional sub-frame grid stacking for higher temporal density.

Claimed capabilities: single/multi-image, video, OCR, PDF parsing, visual grounding, tool calling, and an optional thinking mode (enable_thinking). The card claims a new performance–efficiency Pareto frontier — "higher overall score in less inference time than Qwen3.6 35B-A3B" — but the actual evaluation tables are images not present in the text, so no benchmark numbers are verifiable from the card itself. Treat the Pareto claim as vendor-reported.

What's actually released

A lot: base weights plus FP8, int4, AWQ, GPTQ, and GGUF variants; the DSpark draft model; Ollama availability; day-0 support claimed for SGLang, vLLM, llama.cpp, OpenVINO (Intel), and Rockchip/Houmo/AMD/Arm; iOS/Android/HarmonyOS apps with source in an edge-deployment repo; fine-tuning via SWIFT and LLaMA-Factory plus a "Self-Replay Fine-Tuning" method pitched as anti-forgetting customization (no details or paper in the card). Apache-2.0. Unedited on-device screen recordings (iPhone 17 Pro Max, iPad M4) are offered as evidence.

Caveats

• The 13 tok/s / 3 GB figure comes with unstated quantization bit-width and context length; "lossless" presumably means routing-faithful, not quantization-free. How much the fine-grained quantization costs on, say, video benchmarks is unreported.
• LES throughput should degrade with routing unpredictability — worst-case latency under low prediction accuracy isn't characterized.
• Whether LES is in the open deployment stack or only in OpenBMB's apps isn't crystal clear from the card; check the edge repo before assuming you can reuse it.
• Requires transformers>=5.18.0, and downsample_mode must be passed to both apply_chat_template and generate — an easy silent-bug footgun.

Try it

pip install "transformers[torch]>=5.18.0" torchvision av

from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "openbmb/MiniCPM-V-4.7-35B-A3B"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

messages = [{"role": "user", "content": [
    {"type": "image", "url": "https://huggingface.co/datasets/openbmb/DemoCase/resolve/main/refract.png"},
    {"type": "text", "text": "What causes this phenomenon?"}]}]

mode = "16x"  # "4x" for finer detail (OCR, documents)
inputs = processor.apply_chat_template(messages, tokenize=True, add_generation_prompt=True,
    return_dict=True, return_tensors="pt", downsample_mode=mode, max_slice_nums=36).to(model.device)
out = model.generate(**inputs, downsample_mode=mode, max_new_tokens=512)
print(processor.batch_decode([o[len(i):] for i, o in zip(inputs.input_ids, out)], skip_special_tokens=True)[0])

Model card and quantized variants: https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B. For the phone experience, grab the iOS/Android apps from the linked edge-deployment repo — that's where the LES claim lives or dies.