Paper notes
Diffusion Models Are Real-Time Game Engines
First arXiv submission: · ICLR 2025
Presents GameNGen, a diffusion model that interactively simulates DOOM from previous frames and player actions.
Contribution and setting
Uses agent-collected gameplay, conditioning augmentation, and decoder fine-tuning to sustain autoregressive visual interaction.
- Model output
- The next rendered game frame.
- Action interface
- DOOM player actions.
- State and memory
- Conditioning on a sequence of recent frames and actions; continued play feeds generated frames back into the model.
Relationship to NeuralOS
Interactive-simulation background cited by NeuralOS; NeuralOS also refers to its perceptual-evaluation setup.
Scope and limitations
The simulator is trained and evaluated on DOOM.
Related terms: GameNGen, neural game engine, real-time simulation, diffusion renderer.
Cite this paper
ArXiv preprint
arXiv:2408.14837, initial submission 2024-08-27.