The bibliography
Papers organized by research setting and first arXiv submission date.
Each entry describes the model, its experiments, and its connection to NeuralOS. The downloads contain the same notes and citation data.
Showing 13 papers
No papers match these filters. Try a broader term or reset the filters.
Learns to predict Atari frames from past observations and control inputs, connecting visual generation to an agent's actions.
First arXiv submission 2015-07-31 · NeurIPS 2015
Combines learned visual compression and recurrent dynamics with a controller, including training an agent inside a generated environment.
First arXiv submission 2018-03-27 · arXiv
Learns an interactive visual game simulator from gameplay and keyboard actions using a generative adversarial network.
First arXiv submission 2020-05-25 · CVPR 2020
Proposes an operating-system interaction paradigm in which connected generative models interpret user intentions and mediate or replace conventional application interfaces.
First arXiv submission 2023-10-07 · IEEE CogMI 2023
Learns controllable visual environments from videos without ground-truth action labels, using a learned latent action space.
First arXiv submission 2024-02-23 · arXiv
Provides real computer environments and reproducible tasks spanning desktop applications, websites, and file operations.
First arXiv submission 2024-04-11 · NeurIPS 2024
Introduces DIAMOND, an agent trained in a diffusion world model, and studies how preserving visual detail affects control.
First arXiv submission 2024-05-20 · NeurIPS 2024
Provides a functional Android environment with parameterized tasks and programmatic checks of device state.
First arXiv submission 2024-05-23 · ICLR 2025
Presents GameNGen, a diffusion model that interactively simulates DOOM from previous frames and player actions.
First arXiv submission 2024-08-27 · ICLR 2025
Predicts the effects of candidate web actions as natural-language descriptions of important changes, then uses those predictions for action selection.
First arXiv submission 2024-10-17 · ICLR 2025
Predicts future mobile-app screens to help agents compare possible actions, separating graphic content from textual content.
First arXiv submission 2025-04-15 · arXiv
Studies neural operating systems through learned, interactive GUI simulation: a model produces desktop screens in response to mouse and keyboard events, opening a path toward generative computer interfaces.
First arXiv submission 2025-07-11 · ICLR 2026
Predicts a GUI's next visual state by generating renderable code, and evaluates whether the predicted outcomes help mobile-agent navigation.
First arXiv submission 2026-02-10 · arXiv