Papers · explanations · citations

Neural operating systems
and GUI world models

What if computers generated their interfaces and learned their behavior?

Explore neural operating systems, interfaces generated on the fly, and applications learned from demonstrations. Follow the connections to generative UI, personalized human-computer interaction, GUI world models, and environments for computer-use agents.

Three ways into the literature

Read in sequence, or follow your question.

Research vision and implementation

NeuralOS

First preprint · July 11, 2025
Revised · March 12, 2026
ICLR 2026

Towards Simulating Operating Systems via Neural Generative Models

Luke Rivard, Sun Sun, Hongyu Guo, Wenhu Chen, Yuntian Deng

NeuralOS explores generative computing: interfaces that adapt to the user’s inputs, context, and intentions. The July 2025 paper articulates this vision and develops a learned computer environment, combining recurrent state tracking with a diffusion renderer to generate screens from mouse and keyboard events.

The original vision includes personalized interaction and experiences that cross application boundaries. The March 2026 revision demonstrates learning to simulate launching, playing, and closing a Doom application from synthetic demonstrations, without that application being installed in the recorded desktop environment.

Download the DiT/Mamba pretrained models and bundled runtime to generate 1024 × 768 desktop frames locally. The release includes the latest v6 checkpoint and the v4 checkpoint used by the online demo.

Generative interfaces on the fly

NeuralOS’s original paper asks what computing becomes when models generate the interface as people interact. Its research vision spans personalized controls, interaction through language or gestures, and experiences that connect applications and interactive media. Read the July 2025 introduction.

Personalized, adaptive interfaces

Generate interfaces around a person’s task, context, and intentions. This is a human-computer interaction question about what the interface should become, alongside the modeling question of how to generate it.

Applications learned from examples

Interaction demonstrations describe how an application behaves. NeuralOS’s synthetic Doom experiment shows how a generative model learns a new simulated application from constructed examples.

Explore the vision, experiments, and connections →

How the research connects

GUI world models

A GUI world model predicts how an interface changes after an action. ViMo predicts mobile screens for planning; NeuralOS learns a desktop environment that responds to successive mouse and keyboard events.

Generative GUI simulation

The model generates the interface throughout an interaction: open a window, move it, switch away, and return. NeuralOS connects this simulation problem to neural operating systems and adaptable interfaces, with recurrent state tracking and a diffusion renderer.

A short reading path

NeuralOS → GameNGen → DIAMOND → ViMo → Code2World → OSWorld

Follow the conceptual path above, or browse the full list in chronological order. Each entry describes the model’s predictions, actions, state, and relationship to NeuralOS.

Browse all papers → · Download the bibliography →