National Cyber Warfare Foundation (NCWF)

Vision-Language-Action
(VLA) models generalize semantically well but often lack fine-grained modeling
of world dynamics. We present MotuBrain, a unified World Action Model that
jointly models video and action under a UniDiffuser formulation with a
three-stream Mixture-of-Transformers architecture. A single model supports
policy learning, world modeling, video generation, inverse dynamics, and joint
video-action prediction, while scaling to heterogeneous multimodal data such as
video-only, task-agnostic, and cross-embodiment robot data. Building on Motus,
MotuBrain further introduces unified multiview modeling, an independent text
stream for stronger language-action coupling, a shared cross-embodiment action
representation, and an efficient post-training and deployment recipe for
long-horizon real-world control. Our inference stack combines step reduction,
compilation, FP8 quantization, DiT caching, V2A-style action-only inference,
and real-time chunked closed-loop execution, achieving over 50x speedup over a
naive baseline and up to 11 Hz inference. Experimentally, MotuBrain achieves
95.8% and 96.1% average success on RoboTwin 2.0 under clean and randomized
settings, respectively, attains the strongest reported EWMScore in our
WorldArena comparison, and adapts to new humanoid embodiments with only 50--100
trajectories. These results show that unified world action models can scale in
generality, predictive accuracy, and real-world deployability.

View
Paper

103

Save

Cite

AI Overview

Foundations of
World Action Models

The advancement
of embodied artificial intelligence is increasingly centered on bridging the
gap between high-level semantic understanding and low-level physical control.
Traditional Vision-Language-Action (VLA) policies often rely on pre-trained
vision-language models to map visual scenes and textual instructions to robot
commands. While these models excel at identifying objects and following
linguistic commands, they frequently lack a deep understanding of physical
dynamics—the "common sense" of how objects move and interact over
time. This limitation often results in behaviors that look correct in static
snapshots but fail during complex, contact-rich manipulation.

MotuBrain Architecture Figure 1: The MotuBrain framework
integrates language, video, and action streams into a unified generative model,
enabling it to predict both world dynamics and robot actions.

MotuBrain
addresses this by adopting the framework of a World Action Model (WAM). Instead
of treating action prediction as a simple classification or regression task, a
WAM unifies world modeling (predicting future visual states) and action
generation (deciding how to move) into a single objective. By learning to
predict how the world evolves in response to its actions, the model develops an
internal representation of physics and temporal causality. This approach
leverages the rich spatiotemporal priors found in large-scale video data, which
contain vast amounts of information about object permanence, friction, and
fluid dynamics that are not captured in static image-text datasets.

The MotuBrain
architecture is built on UniDiffuser, a framework that jointly models
multiple modalities. This allows the system to operate in various modes,
including acting as a policy, a world simulator, or an inverse dynamics model.
By unifying these tasks, MotuBrain mitigates the error accumulation common in
two-stage systems, where a separate video predictor might generate physically
impossible futures that lead a downstream controller astray.

Architectural
Design: The Three-Stream Mixture-of-Transformers

At the core of
MotuBrain is a Mixture-of-Transformers (MoT) architecture comprising
three distinct streams: text, video, and action. Each stream is designed to
handle the unique characteristics of its modality while allowing for rich
cross-modal interaction.

Text Stream: This branch processes linguistic
instructions. It acts as a conditioning signal, ensuring that the model's
predictions and actions are aligned with human intent. While it
participates in the self-attention mechanism to inform other streams, it
does not have its own generative output head.

Video Stream: This stream handles high-dimensional visual
latents. It utilizes a pre-trained VAE (Vidu VAE) to encode images into a
latent space. The model then learns to predict the "velocity" of
these latents through flow matching, a generative technique that defines a
smooth path from noise to a clean signal.

Action Stream: Robot actions—such as gripper
positions and joint angles—are treated as continuous signals. Like the
video stream, actions are predicted using flow matching, allowing for the
generation of complex, multi-modal action distributions.

To manage the
computational complexity of these interacting streams, MotuBrain employs an H-bridge
attention design. In this configuration, the bottom $M$ layers and top $N$
layers of the Transformer use decoupled attention, where each modality
processes its own information independently to preserve specific features. The
middle $L$ layers use full joint attention, allowing the text, video, and
action features to fuse deeply. This hybrid approach balances the need for
cross-modal coordination with the efficiency of modality-specific processing.

The generative
process follows a flow-matching objective. For a latent $x$, the model predicts
a velocity field $v(x_t, t)$ that transforms noise $x_1$ into data $x_0$
according to:

$$

x_t = (1-t)x_0 + tx_1

$$

$$

v(x_t, t) = \frac{dx_t}{dt}

$$

This formulation
allows the model to refine both the predicted future video frames and the
accompanying robot actions simultaneously.

The Four-Level
Data Pyramid

A significant
challenge in training embodied models is the scarcity of high-quality robot
interaction data compared to the abundance of internet video. MotuBrain
overcomes this by utilizing a "data pyramid" strategy that
progressively refines the model's knowledge across four levels of data:

Level 1: Internet Videos: Large-scale, general-purpose video
data provides foundational knowledge about world motion and physical laws.

Level 2: Ego-centric Videos: First-person videos of humans
performing tasks provide a bridge between general motion and task-oriented
manipulation.

Level 3: Heterogeneous Robot Data: Data from various robot platforms
and embodiments helps the model learn transferable control regularities
that are not tied to a specific hardware setup.

Level 4: Specific Robot Data: Targeted data from the final
deployment robot ensures the model is perfectly calibrated for its
specific sensors and actuators.

Training Stages and Attention Masks Figure 2: The training process transitions
from pure video pre-training to joint video-action alignment and final
embodiment-specific post-training.

The training
proceeds in two primary stages. In Stage 1, the model focuses on video
generation, adapting internet-scale priors to the domain of robotic
manipulation. In Stage 2, the action branch is introduced and trained
alongside the video branch on robot-specific datasets. During this stage, the
model learns the crucial mapping between visual changes and physical actions.

To further
enhance robustness, MotuBrain uses noisy-conditioning. During training,
the conditioned visual frame (the "current" observation) is slightly
perturbed with noise. This forces the model to rely more on the underlying
scene structure and the linguistic instruction rather than overfitting to
pixel-perfect visual cues, making it more resilient to real-world lighting
changes or camera noise.

Unified
Representations for Embodiment

To enable a
single model to control different types of robots, MotuBrain implements a unified
action representation. Instead of predicting absolute joint angles, which
vary wildly between robot models, it predicts relative end-effector (EEF)
poses.

The relative pose
$P_{rel}$ is calculated by comparing the current pose $P_{curr}$ to a reference
pose $P_{ref}$ (usually from the conditioned frame):

$$

p_{rel} = p_{curr} - p_{ref}

$$

$$

R_{rel} = R_{ref}^{-1} R_{curr}

$$

This
representation allows the model to learn a general "vocabulary" of
movement (e.g., "move forward 10cm") that can be mapped to different
physical hardware with minimal tuning.

Furthermore,
MotuBrain supports multiview inputs naturally. Visual information from
multiple cameras is encoded and concatenated at the token level. To handle the
spatial relationships between these views without changing the model
architecture, the system uses 3D Rotary Positional Embeddings (3D RoPE)
with view-dependent offsets. This allows the model to understand the 3D
geometry of the scene by correlating features across different camera angles.

Optimization
Suite for Real-Time Execution

Large generative
models are often too slow for the high-frequency control loops required in
robotics (typically $10\text{--}100 \text{ Hz}$). MotuBrain implements a
comprehensive suite of optimizations to achieve a $50\times$ speedup, moving
from a latent $0.20 \text{ Hz}$ to a deployable $11.11 \text{ Hz}$.

V2A-Style Inference: The model uses an asymmetric
attention structure where action tokens can attend to video tokens, but
video tokens do not necessarily need to attend to action tokens during all
phases of generation. This allows for an "action-only" suffix in
the denoising process, where the heavy video computations are frozen after
a few steps, and only the lightweight action stream continues to refine.

SNR-Based Time-Shifting: By using different noise schedules
for video and action—specifically a higher $\text{timeshift} = 6$ for
video and $\text{timeshift} = 1$ for action—the model becomes more robust
to visual noise. This enables reducing the number of inference steps from
$50$ to $30$ without losing accuracy.

DiT Caching: Many parts of the world do not change
rapidly between control cycles. The model employs a cache for the
Transformer blocks (similar to DreamZero), skipping redundant evaluations
when the predicted velocities are stable.

Quantization and Compilation: The model uses FP8 quantization
for large linear layers and torch.compile to optimize the execution graph,
significantly reducing memory bandwidth requirements and improving
hardware utilization.

Quantitative
and Qualitative Performance

MotuBrain has
been evaluated on standard simulation benchmarks, world model leaderboards, and
in complex real-world environments.

In the RoboTwin
2.0 simulation benchmark, MotuBrain achieved an average success rate of
$95.8\%$ across $50$ diverse tasks, significantly outperforming previous VLA
and WAM models. Notably, it maintained a high success rate of $96.1\%$ even
under randomized scene conditions, demonstrating strong generalization.

Scaling Curves Figure 3: Performance scaling as a
function of the number of training tasks, showing MotuBrain's superior data
efficiency.

On the WorldArena
leaderboard, which evaluates the quality of world modeling (the
"imagination" of the model), MotuBrain secured the top spot with an
EWMScore of $63.77$. This indicates that it not only acts well but also
generates the most physically realistic visual predictions among existing
embodied models.

WorldArena Leaderboard Figure 4: MotuBrain ranks first on the
WorldArena leaderboard, outperforming specialized video generation models.

Real-world
experiments demonstrated the model's ability to handle complex, long-horizon
tasks such as mixing cocktails, preparing meals, and arranging flowers. These
tasks require both bimanual coordination and the ability to self-correct. For
instance, in the flower arrangement task, the model was observed to
"retry" a failed insertion, showing an inherent understanding of the
task's final goal rather than just mimicking a sequence of movements.

Real World Tasks Figure 5: A humanoid robot performing a
complex cocktail-mixing sequence using MotuBrain.

By combining the
predictive power of video generation with the precision of direct action
control, and optimizing the entire pipeline for real-time deployment, MotuBrain
provides a scalable foundation for general-purpose robotic intelligence.

Motus: A unified latent action world model

This paper
introduces Motus, the direct predecessor to MotuBrain. The authors explicitly
state that MotuBrain builds upon the foundational unified world-action
formulation and architecture established by Motus, making it a critical piece
of context.

Hongzhe Bi,
Hengkai Tan, Shenghao Xie, Zeyuan Wang, Shuhe Huang, Haitian Liu, Ruowen Zhao,
Yao Feng, Chendong Xiang, Yinze Rong, Hongyan Zhao, Hanyu Liu, Zhizhong Su, Lei
Ma, Hang Su, and Jun Zhu. Motus: A unified latent action world model, 2025.

One transformer fits all distributions in
multi-modal diffusion at scale

This paper
introduces UniDiffuser, the core technical framework that MotuBrain adopts to
jointly model the continuous video and action modalities within a single
generative model. This citation is fundamental to understanding the underlying
mechanism of MotuBrain's architecture.

Fan Bao, Shen
Nie, Kaiwen Xue, Chongxuan Li, Shi Pu, Yaole Wang, Gang Yue, Yue Cao, Hang Su,
and Jun Zhu. One transformer fits all distributions in multi-modal diffusion at
scale, 2023.

Vidu: a highly
consistent, dynamic and skilled text-to-video generator with diffusion models

Vidu is the
pre-trained video generation model that serves as the foundation for
MotuBrain's video stream. The paper's method starts from pre-trained Vidu
weights, inheriting its rich spatiotemporal priors from large-scale web video
data.

Fan Bao, Chendong
Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu,
Yaole Wang, and Jun Zhu. Vidu: a highly consistent, dynamic and skilled
text-to-video generator with diffusion models, 2024.

Worldarena: A unified benchmark for
evaluating perception and functional utility of embodied world models

WorldArena is a
key benchmark used to validate MotuBrain's capabilities as a world model. The
paper highlights its top-ranking EWMScore on this benchmark as a primary result
to demonstrate that the model not only acts effectively but also accurately
predicts future world dynamics.

Yu Shang,
Zhuohang Li, Yiding Ma, Weikang Su, Xin Jin, Ziyou Wang, Lei Jin, Xin Zhang,
Yinzhou Tang, Haisheng Su, et al. Worldarena: A unified benchmark for
evaluating perception and functional utility of embodied world models, 2026.

World action models are zero-shot policies

This paper
represents a contemporary World Action Model (WAM) and is directly cited for
several inference optimization techniques, such as DiT caching and action
smoothing, that MotuBrain adopts to achieve real-time performance. It
contextualizes the engineering efforts required to make large WAMs practical
for robotic deployment.

Seonghyeon Ye,
Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel
Indupuru, You Liang Tan, Chuning Zhu, Jiannan Xiang, et al. World action models
are zero-shot policies. arXiv preprint arXiv:2602.15922, 2026.

Audio

MotuBrain: An Advanced World Action Model for Robot Control

0:00 / 6:47

MotuBrain: An Advanced World Action Model for Robot Control

Transcript

Autoresearch

Run Autoresearch

Autoresearch turns this paper into a runnable project: an
agent sets up the project, writes experiments, and runs them on your own
compute or in the cloud.

Run with your local harness

alphaXiv/openresearch-cli

Reproduce paperMarimo
notebook

# Install and start the orx CLI

curl -LsSf https://openresearch.sh/install.sh | shorx up

# Then ask it to reproduce this paper

/reproduce-paper 2604.27792 MotuBrain: An Advanced World
Action Model for Robot Control

MotuBrain: An Advanced World Action Model for Robot Control- Alphaxiv.org Article

Comments