FACT: Failure-Aware Causal Training for World-Action Models

Overview

Recent world-action models (WAMs) show that co-training policies with future prediction can provide physical priors for action generation. Building on the future-prediction ability of video models, many WAMs generate future videos and recover actions with inverse-dynamics models, or use these predicted videos as goal conditions for action generation. In both cases, the world model is trained mostly on successful demonstrations and has little reason to predict the consequences of bad actions. We introduce FACT, a causal World-Action Model that predicts future video and task progress conditioned on the executed action. This action-conditioned interface allows failure rollouts to supervise action consequences, turning bad actions into valid future targets rather than being discarded. Failure-aware training makes the progress predictor aware of both successful and failed action outcomes, which can optionally be used to score sampled action candidates at inference. Extensive experiments on simulation and real-world bimanual manipulation tasks show that FACT outperforms many existing baselines, improves as failure data are incorporated into training, and reduces success-biased future hallucination under bad actions.

Method

One causal diffusion transformer — act, then imagine.

FACT architecture: a shared causal diffusion transformer denoises action, value, and future-video tokens; value and future video condition on the clean action slot G, not the noisy A. Failure rollouts mask the action-imitation loss but keep value and future-video supervision.

Act, then imagine

Value and future video condition on the clean action G — the noisy action slot A never sees it, so future prediction sharpens action generation without leaking targets.

Failures teach consequences

Failure rollouts mask the action-imitation loss but still supervise the observed failed future and a lowered progress value — consequences, not behavior.

Optional action scoring

Trained on failed outcomes, the value head becomes action-sensitive and can rank N sampled candidates at deployment for extra reliability.

Real-world Experiment

Front view

Insert the screwdriver

Zip the tool bag

Pack and close the toolbox

Open the bottle cap

Side view

Insert the screwdriver

Zip the tool bag

Pack and close the toolbox

Open the bottle cap

Stack Cubes

Pick Cubes

Handover

Stack Bowls

Pour

unseen

Pick Cubes — held-out variant

Results

RoboTwin simulation. Average success over 50 tasks; failure co-training lifts FACT to 87.5%, on par with the best baseline while running ~3× faster at deployment.

Real-world bimanual tasks. Failure-aware training improves seen-task success from 82% to 89% (92% with scoring), without any robot pretraining.

Failure-data scaling. Success keeps improving as more failure rollouts are mixed into training — no early saturation.

Less success-biased hallucination. On failure rollouts, future-prediction PSNR jumps by +6.4 dB while success futures stay unchanged.

Qualitative comparison: under the same bad action, the success-only model hallucinates a successful grasp while failure-aware co-training predicts the observed failed outcome. — **Same bad action, two imaginations.** The success-only model hallucinates a successful grasp (dotted box); failure-aware FACT predicts the failure that actually happens.

BibTeX

@article{peng2026fact,
  title   = {FACT: Failure-Aware Causal Training for World-Action Models},
  author  = {Peng, Quanquan and Liang, Yutong and Yan, Rui and Hansen, Nicklas and Wang, Xiaolong},
  journal = {arXiv preprint arXiv:2608.10232},
  year    = {2026}
}