QwenLM/Qwen-AgentWorld

TL;DR: Qwen-AgentWorld flips the usual agent-training setup: instead of training the agent, it trains the environment — an LLM that simulates what an MCP tool server, terminal, Android device, web page, or OS returns when an agent acts. Environment modeling is baked in from continued pretraining onward ("native world model"), and the payoff is that you can run agent RL against the simulator instead of real infrastructure, inject controlled perturbations, or invent entirely fictional worlds — with reported downstream gains that transfer to real benchmarks. Weights (35B MoE, 3B active, Apache-2.0) and a seven-domain benchmark are released.

What it is

A "language world model" (LWM): given a trajectory of agent actions, predict the next environment observation via long chain-of-thought — the JSON a tool call would return, the stdout of a bash command, the DOM state after a click. It covers seven domains in one model: MCP, Search, Terminal, SWE, Android, Web, OS. The claimed novelty is that this isn't a fine-tuned add-on: a three-stage pipeline (CPT injects environment knowledge, SFT activates next-state-prediction reasoning, RL sharpens simulation fidelity) trains on 10M+ real interaction trajectories with environment modeling as the objective from the start.

Why the results are interesting (if they hold)

Three application results stand out beyond raw simulation quality:

1. Environment scaling. Running Sim RL on 4k out-of-distribution "OpenClaw" environments simulated by the 397B LWM lifts Qwen3.5-35B by +4.3 on Claw-Eval and +7.1 on QwenClawBench — while using a generic frontier model (Qwen3.6-Plus) as the simulator gives essentially nothing (+1.3 / −0.1). That's the key control: a strong general LLM is not a good enough environment simulator; dedicated world-model training matters.

2. Controllable and fictional simulation. Injecting targeted perturbations via control instructions during Sim RL gives +12.3 on MCPMark vs. +3.1 for uncontrolled Sim RL. Training search agents in fully invented, self-consistent worlds transfers to real WideSearch (+16.3 F1 item at 35B scale). The claim that fictional-world training can surpass real-environment training is the most provocative bit — worth reading the tech report for.

3. World modeling as agent warm-up. RL on single-turn next-state prediction (non-agentic data) transfers to multi-turn tool-calling: +6.3 Terminal-Bench 2.0, +12.8 WideSearch, +9.0 BFCL v4, including three fully out-of-domain benchmarks. Predicting environments apparently teaches something useful about acting in them.

What's actually released

• Qwen-AgentWorld-35B-A3B weights on HF/ModelScope (MoE, 3B active, 256K context, Apache-2.0). Note: the 397B-A17B variant tops the tables but is not in the release list — only the 35B appears to be open.
• AgentWorldBench: per-domain JSONL, ~7 test files, scored on a five-dimension rubric (Format, Factuality, Consistency, Realism, Quality) by an LLM judge; eval scripts and judge prompts in the repo.
• Domain-specific world-model system prompt templates in prompts/.
• No training code or the 10M trajectory corpus, despite the triage note suggesting trajectories were released — README only mentions weights and benchmark.

On AgentWorldBench, the 35B scores 56.39 overall (+8.66 over its base) and the 397B claims 58.71, edging GPT-5.4 (58.25). Caveat: this is their own benchmark, scored by an LLM judge on rubric dimensions like "realism," and all numbers are self-reported. The margins over frontier models are small; the more convincing evidence is the downstream Sim RL transfer, not the leaderboard.

Practicalities and gaps

Serving needs tensor-parallel-size 4 in their examples (a 35B MoE in bf16 is ~70GB, so 4×A100/H100 or 2 with quantization). A vLLM quirk: the checkpoint has vision module definitions but no vision weights, so --language-model-only is required. Simulation fidelity on genuinely novel tool semantics is the thing to stress-test before trusting it as an RL environment — a world model that hallucinates plausible-but-wrong observations could teach agents plausible-but-wrong behavior, and the README doesn't quantify fidelity outside its own rubric. Fine-tuning is "use Swift/Llama-Factory/UnSloth" — no first-party recipes.

Try it

vllm serve Qwen/Qwen-AgentWorld-35B-A3B \
  --port 8000 --tensor-parallel-size 4 \
  --max-model-len 262144 --reasoning-parser qwen3 \
  --language-model-only --trust-remote-code

Then prompt it as an environment: system prompt from prompts/terminal/system_prompt.txt, user message Action: execute_bash\nCommand: ls -la /home/user/project/, and it returns the simulated terminal output. Benchmark data: huggingface-cli download Qwen/AgentWorldBench --repo-type dataset. Paper: arXiv:2606.24597.