AlayaLab/Evoke

TL;DR: EVOKE is an interactive video world model that answers the long-horizon memory problem not with a bigger context window but with an external, camera-indexed 3D world-state bank: emitted frames are depth-estimated, unprojected into a persistent point cloud, and re-rendered into whatever view the camera needs next. Combined with a 3-step, CFG-free distilled student (1.5 s of 384×640 video per 2.11 s on one H200) and per-chunk re-prompting mid-rollout, this is the most complete "endless world" release I've seen — code, four checkpoints, teacher weights, demos, and a local Director UI, all Apache-2.0 (with one licensing catch).

The idea

Every autoregressive video world model faces the same trade: remember more of the session and your context (and compute) grows without bound, or window the context and forget where you've been. EVOKE moves scene geometry out of the denoiser entirely. The bank has three operations: write (a monocular depth model — ViGeo, vendored — estimates depth for each emitted chunk under known poses and unprojects it into a persistent point cloud), read (the current camera pose addresses the bank, ranks sources by co-visibility, fuses up to eight, and a batched z-buffered scatter returns a warped image plus per-pixel visibility mask), and evict (an optional retention window for hour-scale runs). The denoiser's context stays bounded no matter how long the session runs; only what the current view needs is retrieved.

Inside the denoiser, each latent chunk is laid out along the RoPE frame axis as a tiered memory:

prefix | long(16) | mid(2) | warp(W) | prev_short(1) | noise(W)

prefix is the frame-0 global anchor; long/mid are parametric history compressed only by coarser patchify kernels (long is (4,8,8), everything near the noise is (1,2,2)); warp is the world-state bank rendered into the current view, with RoPE indices overlapping the noise window; prev_short is the last latent of the previous chunk. Geometric persistence comes from the bank; short-range continuity from the tiers.

The training story

The name of the paper is the tell: the contribution is as much about supervision as inference. Long-horizon self-forcing distillation is usually unaffordable because the teacher's memory/compute grows quadratically with rollout length. EVOKE rebuilds the teacher with chunk-wise grouping, distant-frame retrieval, and a linear-attention global state so both grow linearly — which is what makes 30 s self-forced supervision tractable. The pipeline is four stages: a 50-step CFG-5.0 camera-controllable model (built on LingBot-World, with Wan-lineage VAE/text encoder via Helios), few-step distillation to a 3-step pyramid, 30 s long-video distillation against a dual-expert DMD teacher, then a short post-distillation back against the stage-1 teacher to firm up camera control. All four checkpoints plus the teacher experts ship on HF.

What's actually there, and caveats

Code, weights, training launchers (each resuming from its released checkpoint — pretraining data is not released, and the bundled configs point at a single 60 s pipeline-check clip), inference scripts for t2v/i2v/v2v/mid-rollout re-prompting, and a Director Web UI with dual-stick camera control, streaming previews, and revision branching. Notably honest touches: the distilled models were trained on v2v conditioning only, so i2v/t2v on them is flagged as zero-shot; demos are stated as not cherry-picked across seeds.

Caveats. The WBench SOTA claim is a day-0 self-reported snapshot with no numbers in the README — verify independently. Everything downstream depends on monocular depth quality; drift in the point cloud over long rollouts is the obvious failure mode to probe. Both depth backends are CC-BY-NC-4.0, which poisons commercial use despite the Apache-2.0 repo. The warp/attention recipe must match between training and inference or quality silently degrades. Requirements are strict: Python 3.10, CUDA 12.4, torch 2.4, flash-attn, a non-PyPI diffusers fork, and the 2.11 s/chunk figure is on an H200.

Try it

pip install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt   # install the pinned diffusers fork first

hf download AlayaLab/Evoke --local-dir models
hf download pkqbajng/ViGeo --local-dir models/ViGeo1.1   # required depth backend

# 30 s rollout from a first frame + pose track, 3 steps, no CFG
MODE=i2v NUM_CHUNKS=20 bash scripts/inference/infer_post_distill.sh

# or the interactive Director UI
python -m pip install -r ui/requirements.txt && python ui/app.py

If you only look at one thing, run the MODE=segment re-prompting case (aurora igniting mid-rollout at chunk 3 of 6) and then push a v2v rollout well past 20 chunks with a camera loop to see whether the bank actually holds the scene together when you return to where you started.