# Neural operating systems Study how learned generative models can supply computer interaction, interface behavior, and adaptable user experiences. The research direction includes conceptual proposals and learned interface simulators. Papers are ordered by first public date. For example: A user double-clicks an icon and a learned simulator predicts the subsequent desktop frames and window state. Version 1.0.1; literature checked through 2026-10-19. [Topic guide](https://neural-os.com/reading-list/guides/neural-operating-systems/) · [JSON](https://neural-os.com/reading-list/data/topics/neural-operating-systems.json) ## Papers — oldest first ## Prompt-to-OS (P2OS): Revolutionizing Operating Systems and Human-Computer Interaction with Integrated AI Generative Models Authors: Gabriele Tolomei, Cesare Campagnano, Fabrizio Silvestri, Giovanni Trappolini. Proposes an operating-system interaction paradigm in which connected generative models interpret user intentions and mediate or replace conventional application interfaces. First public: 2023-10-07. ArXiv initial submission: 2023-10-07. [Original paper](https://arxiv.org/abs/2310.04875) · [Reading-list entry](https://neural-os.com/reading-list/papers/2310.04875/) [Preprint BibTeX](https://neural-os.com/reading-list/papers/2310.04875/citation.bib) · [Preprint CSL-JSON](https://neural-os.com/reading-list/papers/2310.04875/citation.csl.json) · [JSON record](https://neural-os.com/reading-list/papers/2310.04875/record.json) **Contribution:** Sets out a broader vision for natural-language, personalized computer interaction, including language and diffusion models as interface components. **Model output:** Envisioned context-dependent responses and generated interfaces. **Action interface:** Natural-language user requests, including text and speech. **State and memory:** Proposes personalization and contextual interaction. **Relationship to NeuralOS:** NeuralOS cites Prompt-to-OS as background for generative computer interaction. **Scope and limitations:** A conceptual proposal for generative computer interfaces. [Relationship source](https://arxiv.org/html/2507.08800v2) Related terms: Prompt-to-OS, generative interface, neural operating system, human-computer interaction. ### Approaches - [Neural operating systems](https://neural-os.com/reading-list/guides/neural-operating-systems/): Study how learned generative models can supply computer interaction, interface behavior, and adaptable user experiences. The research direction includes conceptual proposals and learned interface simulators. Reading-list version 1.0.1; literature checked through 2026-10-19. ## NeuralOS: Towards Simulating Operating Systems via Neural Generative Models Authors: Luke Rivard, Sun Sun, Hongyu Guo, Wenhu Chen, Yuntian Deng. Studies neural operating systems through learned, interactive GUI simulation: a model produces desktop screens in response to mouse and keyboard events, opening a path toward generative computer interfaces. First public: 2025-07-11. ArXiv initial submission: 2025-07-11. [Original paper](https://arxiv.org/abs/2507.08800) · [Reading-list entry](https://neural-os.com/reading-list/papers/2507.08800/) [Preprint BibTeX](https://neural-os.com/reading-list/papers/2507.08800/citation.bib) · [Preprint CSL-JSON](https://neural-os.com/reading-list/papers/2507.08800/citation.csl.json) · [JSON record](https://neural-os.com/reading-list/papers/2507.08800/record.json) **Contribution:** Formalizes OS-interface generation and develops state tracking, precise cursor conditioning, staged training, and data collection. The 2026 revision also demonstrates learning a new simulated application from synthesized training data. **Model output:** Autoregressively generated desktop screen frames. **Action interface:** Cursor coordinates, left/right clicks, and keyboard press/release events. **State and memory:** A hierarchical recurrent model carries state across frames and conditions a diffusion renderer. **Relationship to NeuralOS:** Introduces NeuralOS and its formulation of learned OS-interface simulation. **Scope and limitations:** Fine-grained typing and long-horizon state retention remain difficult. The simulated interface does not provide the underlying filesystem, processes, or application services of a conventional OS. [Relationship source](https://arxiv.org/html/2507.08800v1) Dated versions: [Original version](https://arxiv.org/html/2507.08800v1) · [Revised version](https://arxiv.org/html/2507.08800v2) [Code](https://github.com/yuntian-group/neural-os) Related terms: NeuralOS, neural operating system, learned computer simulator, generative interface, GUI world model, desktop world model, action-conditioned screen generation. ### Method comparisons **A game world model evaluated on desktop interactions** DIAMOND learns visual game dynamics with a diffusion model. NeuralOS adapts DIAMOND to OS screen prediction as a baseline in Appendix A of its 2026 revision. NeuralOS adds a recurrent state representation and spatial cursor conditioning for desktop interaction. Sources: [DIAMOND paper](https://arxiv.org/abs/2405.12399) · [NeuralOS Appendix A](https://arxiv.org/html/2507.08800v2#A1) Paper details: [Diffusion for World Modeling: Visual Details Matter in Atari](https://neural-os.com/reading-list/papers/2405.12399/) · [NeuralOS: Towards Simulating Operating Systems via Neural Generative Models](https://neural-os.com/reading-list/papers/2507.08800/) **Direct frame generation and renderable-code prediction** NeuralOS predicts desktop frames using learned state and a diffusion renderer. Code2World predicts a UI through code that is then rendered into a screen, and tests its usefulness for mobile navigation. Code2World cites NeuralOS in Section 2.2. Sources: [Original NeuralOS paper](https://arxiv.org/html/2507.08800v1) · [Code2World Section 2.2](https://arxiv.org/html/2602.09856v1#S2.SS2) Paper details: [NeuralOS: Towards Simulating Operating Systems via Neural Generative Models](https://neural-os.com/reading-list/papers/2507.08800/) · [Code2World: A GUI World Model via Renderable Code Generation](https://neural-os.com/reading-list/papers/2602.09856/) Date note: Initial arXiv submission on July 11, 2025; revised on March 12, 2026 and published at ICLR 2026. [Source](https://arxiv.org/abs/2507.08800). ### Published version NeuralOS: Towards Simulating Operating Systems via Neural Generative Models. Luke Rivard, Sun Sun, Hongyu Guo, Wenhu Chen, Yuntian Deng. International Conference on Learning Representations, 2026. [Publication](https://proceedings.iclr.cc/paper_files/paper/2026/hash/2174a6749c3f769358cd21ade1221b34-Abstract-Conference.html) · [BibTeX](https://neural-os.com/reading-list/papers/2507.08800/publication.bib) · [CSL-JSON](https://neural-os.com/reading-list/papers/2507.08800/publication.csl.json) ### Approaches - [Neural operating systems](https://neural-os.com/reading-list/guides/neural-operating-systems/): Study how learned generative models can supply computer interaction, interface behavior, and adaptable user experiences. The research direction includes conceptual proposals and learned interface simulators. - [Interactive learned environments](https://neural-os.com/reading-list/guides/games-to-computers/): Learn how an environment changes after actions, then generate observations during continued interaction. Compare visual fidelity, control interfaces, memory, and the reliability of repeated predictions. - [Environments and evaluation](https://neural-os.com/reading-list/guides/evaluation/): Test action response, visual quality, remembered state, and completed tasks separately. Real desktop and mobile environments provide execution-based checks that complement learned-simulator measurements. Reading-list version 1.0.1; literature checked through 2026-10-19.