Robbyant/lingbot-world-v2

TL;DR: LingBot-World 2.0 ("LingBot-World-Infinity") is a 14B interactive video world model — image + text + per-frame actions in, playable video out — whose headline claims are an unbounded interaction horizon via causal pretraining with a streaming KV cache, and a distilled variant fast enough for 720p/60fps real-time play. Built on Wan2.2, weights for the distilled causal-fast model are actually out; the README is a launch page, though, with essentially no quantitative evidence.

What it is

This is the second iteration of LingBot-World (the team names — LingGuang, Yujun Shen et al. — point at the Ant Group research ecosystem). The pitch is a Genie-3-style playable world model, but with four upgrades over v1: no fixed rollout length, real-time responsiveness, a wider action vocabulary (attacking, archery, spell-casting, shooting, plus text-driven events), and an "agentic harness" where LLM agents drive the world rather than a human.

The interesting engineering is the first two. Most video-diffusion world models generate bidirectionally over a fixed clip, so "interactive" means stitching windows and watching coherence decay. Here the base model is causally pretrained, and inference runs chunk-by-chunk with KV caching — the CLI flags give away the mechanism: --local_attn_size 18 --sink_size 6. That's the StreamingLLM attention-sink recipe transplanted to a video DiT: keep the first 6 chunks as permanent anchors, attend locally over the last 18, and the cache stays constant-size while the rollout runs indefinitely. "Unbounded" is thus an architectural claim about memory footprint, not a proof of unbounded coherence — off-screen content beyond the local window presumably has to be reconstructed from the sink tokens or hallucinated.

The real-time variant is a distillation of the causal base (few-step, presumably consistency/DMD-style — the report is on arXiv but the README doesn't say), claimed sufficient for 720p at 60fps in their serving stack.

The agentic harness

The most novel framing is using the world model as a substrate for agents rather than humans: a pilot agent plans and executes character behavior, while a director agent injects new environmental elements as the scene unfolds — a generated open world that authors itself. This is the "world models as agent training environments" direction made concrete. The README gives zero implementation detail on it, though; that's presumably in the tech report (arXiv 2607.07534).

What's actually released

• Code: inference only (generate.py with KV caching), forked from Wan2.2. No training code, and explicitly no deployment/serving code — they point you at SGLang or flashdreams for that.
• Weights: lingbot-world-v2-14b-causal-fast on HF and ModelScope. The causal-pretrained 14B, bidirectional 14B, and both 1.3B models are all TODO.
• Evidence: demo videos on the project page and hosted real-time demos (Reactor internationally, LingGuang mobile in China). No benchmarks, no numbers, no comparisons in the README — the 60fps/720p and quality-consistency claims are unverifiable from the repo alone. Note the example command runs at 480p across 8 GPUs, and the README admits the hosted demos aren't "full capability."
• License: CC BY-NC-SA 4.0 — non-commercial, viral. Weights are not usable in products.

Assessment

The causal-streaming design with sink tokens is a credible and probably load-bearing idea — expect it to become the standard recipe for long-horizon world models, the same way attention sinks did for streaming LLMs. The things to be skeptical about: how fast identity/geometry drifts once content leaves the 18-chunk window, and whether "60fps" survives outside their internal serving stack (the released code path is 8×GPU torchrun offline inference, not a real-time client). The agentic harness is the most forward-looking piece but is currently paper-ware. 1,570 stars in under a week says the field agrees this is the week's landmark world-model release.

Try it

Fastest path: the Reactor web demo linked from the project page. To run locally (multi-GPU):

git clone https://github.com/robbyant/lingbot-world-v2.git
cd lingbot-world-v2
pip install -r requirements.txt   # torch >= 2.4.0
pip install flash-attn --no-build-isolation
huggingface-cli download robbyant/lingbot-world-v2-14b-causal-fast \
  --local-dir ./lingbot-world-v2-14b-causal-fast

torchrun --nproc_per_node=8 generate.py --task i2v-A14B --size 480*832 \
  --ckpt_dir lingbot-world-v2-14b-causal-fast \
  --image examples/03/image.jpg --action_path examples/03 \
  --dit_fsdp --t5_fsdp --ulysses_size 8 \
  --frame_num 361 --local_attn_size 18 --sink_size 6 \
  --prompt "A serene lakeside scene..."