TL;DR: We present an efficient RL framework for running single-file LLM-generated JavaScript games. These video-game environments can be playtested by humans easily on a browser and can train pixel-based agents at over 1M decisions per second on one GPU node, measured over our 24-game suite of 16 ProcGen and 8 Atari clones.
While many video-game environments (VGEs) have played crucial roles in advancing reinforcement learning (RL), developing novel VGEs or modifying existing ones to support new features has been a laborious process requiring extensive hand-coding. Here we present PlayTrain, an RL framework that combines the abilities of large language models (LLMs) to robustly generate JavaScript (JS) games from a minimal human prompt, and an efficient pipeline that can run any JS game in a standard gym environment. Not only are recent LLMs particularly good at writing JS code, but the JS format also allows users to easily play generated VGEs, while PlayTrain enables us to train RL agents on the exact same games. We demonstrate multiple use cases of PlayTrain, including cloning well-known Atari and ProcGen games in simple JS, where PlayTrain trains pixel-based agents end-to-end at over 1M agent-decisions per second on a single GPU node; and creating modified versions thereof (e.g., that support novel test sets, procedural generation logics, or game dynamics). Through PlayTrain, we reimagine RL VGE development: all we need is a single JS file, generated and modified through an LLM. We discuss promising future RL research directions that PlayTrain unlocks.
Attached here are the eight games the human study covered. Each panel here contains a participant’s best recorded round paired with both IMPALA and PPO trainers. All of them run on the same study’s seeds 2000-step cap horizon. Underneath the videos are the mean arg-max return values over 30 episodes, ten on each of the three seeds the curve beside them plots. Crucially, everyone here plays the exact same game, and human gameplay’s recordings are simply read off JSON files that contain the seed and keyboard action inputs given the step() index.
arrow keys and spacebarkeyboard only · not playable on a touch screen
Every environment with the best IMPALA and the best PPO checkpoint side by side. Videos presented showcase trainer’s best argmax episodes among six and are attached with the average return among those six episodes.
@misc{truong2026playtrainefficientreinforcementlearning,
title={PlayTrain: An Efficient Reinforcement Learning Framework for LLM-Generated Adaptable JavaScript Games},
author={Ryan Truong and Lance Ying and Samuel J. Gershman and Kazuki Irie},
year={2026},
eprint={2609.09059},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2609.09059},
}