Paper notes
NeuralOS: Towards Simulating Operating Systems via Neural Generative Models
First arXiv submission: · ICLR 2026
Initial arXiv submission on July 11, 2025; revised on March 12, 2026 and published at ICLR 2026. Submission history ↗
Original version ↗ · Revised version ↗
Studies neural operating systems through learned, interactive GUI simulation: a model produces desktop screens in response to mouse and keyboard events, opening a path toward generative computer interfaces.
Contribution and setting
Formalizes OS-interface generation and develops state tracking, precise cursor conditioning, staged training, and data collection. The 2026 revision also demonstrates learning a new simulated application from synthesized training data.
- Model output
- Autoregressively generated desktop screen frames.
- Action interface
- Cursor coordinates, left/right clicks, and keyboard press/release events.
- State and memory
- A hierarchical recurrent model carries state across frames and conditions a diffusion renderer.
Relationship to NeuralOS
Introduces NeuralOS and its formulation of learned OS-interface simulation.
Scope and limitations
Fine-grained typing and long-horizon state retention remain difficult. The simulated interface does not provide the underlying filesystem, processes, or application services of a conventional OS.
Related terms: NeuralOS, neural operating system, learned computer simulator, generative interface, GUI world model, desktop world model, action-conditioned screen generation.
Connections between methods
A game world model evaluated on desktop interactions
DIAMOND learns visual game dynamics with a diffusion model. NeuralOS adapts DIAMOND to OS screen prediction as a baseline in Appendix A of its 2026 revision. NeuralOS adds a recurrent state representation and spatial cursor conditioning for desktop interaction.
Direct frame generation and renderable-code prediction
NeuralOS predicts desktop frames using learned state and a diffusion renderer. Code2World predicts a UI through code that is then rendered into a screen, and tests its usefulness for mobile navigation. Code2World cites NeuralOS in Section 2.2.
Cite this paper
Published version
Luke Rivard, Sun Sun, Hongyu Guo, Wenhu Chen, Yuntian Deng. NeuralOS: Towards Simulating Operating Systems via Neural Generative Models. International Conference on Learning Representations, 2026.
Publication BibTeX · Publication CSL-JSON
ArXiv preprint
arXiv:2507.08800, initial submission 2025-07-11.