# Genie: Generative Interactive Environments

Authors: Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feryal Behbahani, Stephanie Chan, Nicolas Heess, Lucy Gonzalez, Simon Osindero, Sherjil Ozair, Scott Reed, Jingwei Zhang, Konrad Zolna, Jeff Clune, Nando de Freitas, Satinder Singh, Tim Rocktäschel.

Learns controllable visual environments from videos without ground-truth action labels, using a learned latent action space.

First public: 2024-02-23.
ArXiv initial submission: 2024-02-23.

[Original paper](https://arxiv.org/abs/2402.15391) · [Reading-list entry](https://neural-os.com/reading-list/papers/2402.15391/)
[Preprint BibTeX](https://neural-os.com/reading-list/papers/2402.15391/citation.bib) · [Preprint CSL-JSON](https://neural-os.com/reading-list/papers/2402.15391/citation.csl.json) · [JSON record](https://neural-os.com/reading-list/papers/2402.15391/record.json)

**Contribution:** Combines video tokenization, a dynamics model, and latent-action learning so users can interact with worlds initialized from images.

**Model output:** Action-conditioned future video frames.

**Action interface:** Discrete latent actions inferred during training rather than recorded physical input events.

**State and memory:** A spatiotemporal dynamics model conditions predictions on previous visual tokens and actions.

**Relationship to NeuralOS:** Cited in NeuralOS's related work. Genie learns its action space from video; NeuralOS takes mouse and keyboard events as inputs.

**Scope and limitations:** Controls use learned latent actions inferred from video.

[Relationship source](https://arxiv.org/html/2507.08800v2)

Related terms: Genie, generative interactive environments, latent actions, foundation world model.

## Approaches

- [Interactive learned environments](https://neural-os.com/reading-list/guides/games-to-computers/): Learn how an environment changes after actions, then generate observations during continued interaction. Compare visual fidelity, control interfaces, memory, and the reliability of repeated predictions.

Reading-list version 1.0.1; literature checked through 2026-10-19.