← 返回 2026-01-07

NitroGen: 用于通用游戏代理的开放基础模型 NitroGen: An Open Foundation Model for Generalist Gaming Agents

Loïc Magne, Anas Awadalla, Guanzhi Wang, Yinzhen Xu, Joshua Belofsky, Fengyuan Hu, Joohwan Kim, Ludwig Schmidt, Georgia Gkioxari, Jan Kautz, Yisong Yue, Yejin Choi, Yuke Zhu, Linxi "Jim" Fan 📅 2026-01-04 👍 46 2026-07-13 08:35
多任务学习 强化学习 游戏AI 行为克隆 视觉-动作模型
arXiv HuggingFace PDF

从40,000小时游戏视频训练的跨游戏通用AI代理

前置知识

行为克隆

通过模仿专家演示数据来学习策略的监督学习方法。在游戏场景中，模型输入当前观察（如屏幕图像），输出对应的玩家操作（如按键、摇杆位置）。目标是学习一个策略函数 $\pi(a|s)$ 使得 $\pi(a|s) \approx \pi_{\text{expert}}(a|s)$。通过最小化预测动作与真实动作之间的差异（如MSE损失）来训练。

本文采用行为克隆而非强化学习，避免了设计奖励函数的困难，能够直接从大规模互联网视频中学习。

流匹配

一种生成模型训练方法，通过学习从噪声分布到目标分布的向量场来生成样本。与扩散模型类似，但训练更稳定。给定数据分布 $p_1$ 和噪声分布 $p_0$，目标是学习向量场 $v_\theta(x,t)$ 使得沿着该场从噪声到数据。条件流匹配则根据条件 $c$ 生成样本，本文中条件是游戏观察图像。

NitroGen使用流匹配来生成连续的动作序列，相比传统方法能更好地处理连续的动作空间。

Gymnasium API

OpenAI Gym的继承者，是强化学习环境的标准化接口，定义了环境的观察空间、动作空间、重置和步进方法。通过统一的API，同一个智能体可以在不同环境中运行，便于算法比较和迁移。观察通常包括图像、状态向量等，动作可以是离散或连续的。

本文的通用模拟器为商业游戏提供了Gymnasium接口，使得研究者可以用标准化方式在不同游戏中训练和测试代理。

研究动机

当前游戏AI领域面临三大困境：强化学习方法虽然能在单个游戏中超越人类（如AlphaStar在StarCraft II），但需要专门的模拟器、设计奖励函数，且训练成本极高，难以扩展到数百个游戏；基于大语言模型的方法（如Voyager）能够处理复杂任务，但依赖于手工制作的程序化API来暴露游戏内部状态，需要复杂的领域特定设计；现有的行为克隆方法（如VPT、SIMA）主要依赖昂贵的人工演示数据，受数据收集成本限制，只能训练少数几个游戏。缺乏大规模、多样化、带标注动作的数据集，阻碍了通用游戏代理的发展。

本文的目标是本文旨在构建一个能够跨越数百个游戏、无需手工设计奖励函数或API、能够从零样本或少量样本中适应新游戏的通用游戏代理。具体目标包括：(1) 收集并标注大规模的互联网游戏视频数据集，(2) 设计统一的跨游戏评估基准，(3) 训练一个基础的视觉-动作模型作为预训练起点，证明互联网规模预训练的可行性。

与已有工作不同的是，本文的独特切入角度是利用一种新的数据源：游戏主播使用的输入覆盖层软件会在屏幕角落实时显示玩家操作（如游戏手柄的按钮状态和摇杆位置）。通过自动解析这些覆盖层，可以从公开的游戏视频中提取动作标签，无需昂贵的人工标注。这种数据源天然包含了真实玩家的多样化行为，且可以扩展到互联网规模。同时，本文开发的通用模拟器能够将任何商业游戏包装成Gymnasium环境，实现了标准化接口。

核心方法

NitroGen的整体思路是先从互联网收集大规模带动作标签的游戏视频，然后训练一个通用的视觉-动作模型，最后通过微调适应新游戏。技术路线分为三个组件：(1) 数据集构建：从公开视频中自动提取玩家操作，构建40,000小时的视频-动作数据集；(2) 评估环境：开发通用模拟器，将商业游戏包装成统一的Gymnasium接口；(3) 模型训练：使用流匹配训练视觉-动作transformer，支持跨游戏泛化。预训练模型可以直接在新游戏上零样本运行，也可以用少量数据微调获得更好性能。

核心创新点是利用输入覆盖层作为动作标签来源。这种数据源有三个关键优势：第一，覆盖层直接显示玩家的真实输入，无需通过逆动力学推断；第二，覆盖层存在于大量公开游戏视频中，可以自动收集到互联网规模；第三，覆盖层包含真实的玩家行为多样性，涵盖不同技能水平和策略。相比需要手工演示的传统方法，这种数据收集方式几乎零成本且可无限扩展。另一个核心创新是通用模拟器，通过拦截游戏引擎的系统时钟来控制仿真时间，实现了对任意商业游戏的程序化控制，无需修改游戏代码。

方法步骤详情

数据集构建分为三个阶段：(1) 模板匹配阶段：采样25帧，使用SIFT和XFeat特征匹配约300个控制器模板，通过关键点对齐估计仿射变换，要求至少20个内点，定位游戏手柄区域；(2) 动作解析阶段：使用SegFormer分割模型处理连续两帧（拼接在空间维度），输出11×11网格上的摇杆位置分割掩码和二进制按钮状态；(3) 质量过滤阶段：只保留50%时间戳有非零按钮或摇杆动作的片段，过滤掉55%的数据，同时在训练时遮罩屏幕上的控制器防止模型利用捷径。模型训练使用单帧256×256的RGB图像作为上下文，生成16动作块，通过AdamW优化器、WSD学习率调度（稳定阶段0.0001）、指数移动平均（衰减0.9999）进行训练。

技术新颖性

技术新颖性体现在三个方面：(1) 数据收集层面，首次利用输入覆盖层构建互联网规模的动作标注数据集，解决了动作标注的瓶颈问题；(2) 环境层面，通用模拟器通过系统时钟拦截实现了对任意商业游戏的标准化控制，无需游戏引擎的专用接口；(3) 模型层面，采用流匹配训练视觉-动作模型，将扩散模型从图像生成迁移到动作生成领域，并证明了仅用单帧上下文和16动作块就能实现良好的时间一致性。相比VPT仅限于Minecraft、SIMA需要人工演示，NitroGen在游戏多样性和数据规模上都有质的提升。

NitroGen overview. NitroGen consists of three main components: (1) Multi-game foundation agent (center) - a generalist vision-action model that takes in game observations and generates gamepad actions, enabling zero-shot gameplay across multiple titles and serving as a foundation for fine-tuning on new games; (2) Universal simulator (left) - an environment wrapper that allows any commercial game to be controlled through a Gymnasium API; and (3) Internet-scale dataset (right) - the largest and most diverse open-source gaming dataset curated from 40,000 hours of publicly available gaming videos, spanning more than 1,000 games with extracted action labels.
Figure 1: NitroGen overview. NitroGen consists of three main components: (1) Multi-game foundation agent (center) - a generalist vision-action model that takes in game observations and generates gamepad actions, enabling zero-shot gameplay across multiple titles and serving as a foundation for fine-tuning on new games; (2) Universal simulator (left) - an environment wrapper that allows any commercial game to be controlled through a Gymnasium API; and (3) Internet-scale dataset (right) - the largest and most diverse open-source gaming dataset curated from 40,000 hours of publicly available gaming videos, spanning more than 1,000 games with extracted action labels.
Video-action dataset pipeline overview. We extract actions from on-screen displays which show the gamepad actions of the player in real-time; called "input overlays". (a) Dataset curation. We collect publicly available videos displaying a "gamepad overlay". The diversity of these overlays presents significant challenges, as gamepads vary widely across content creators in controller types (e.g., Xbox, PlayStation, or others), transparency levels, and visual artifacts introduced by video compression. (b) Action extraction. For each collected video, we localize the gamepad by sampling 25 frames and running keypoint matching against a curated set of templates using SIFT and XFeat features. We use the template-matching results to localize and crop the gamepad region from each video. A hybrid classification–segmentation network is then trained to predict joystick positions and button states from the cropped controller images, enabling accurate reconstruction of player inputs.
Figure 2: Video-action dataset pipeline overview. We extract actions from on-screen displays which show the gamepad actions of the player in real-time; called "input overlays". (a) Dataset curation. We collect publicly available videos displaying a "gamepad overlay". The diversity of these overlays presents significant challenges, as gamepads vary widely across content creators in controller types (e.g., Xbox, PlayStation, or others), transparency levels, and visual artifacts introduced by video compression. (b) Action extraction. For each collected video, we localize the gamepad by sampling 25 frames and running keypoint matching against a curated set of templates using SIFT and XFeat features. We use the template-matching results to localize and crop the gamepad region from each video. A hybrid classification–segmentation network is then trained to predict joystick positions and button states from the cropped controller images, enabling accurate reconstruction of player inputs.
Distribution of the NitroGen dataset across games and genres. After filtering, the NitroGen dataset contains 40,000 hours of gameplay videos spanning more than 1,000 games. (a) Hours per game shows broad coverage, with 846 games having over one hour of data, 91 games with over 100 hours, and 15 games exceeding 1,000 hours each. (b) Genre distribution reveals Action-RPG games are most common (34.9% of total hours), followed by Platformer (18.4%) and Action-Adventure (9.2%) games, with the remainder distributed across seven genres.
Figure 3: Distribution of the NitroGen dataset across games and genres. After filtering, the NitroGen dataset contains 40,000 hours of gameplay videos spanning more than 1,000 games. (a) Hours per game shows broad coverage, with 846 games having over one hour of data, 91 games with over 100 hours, and 15 games exceeding 1,000 hours each. (b) Genre distribution reveals Action-RPG games are most common (34.9% of total hours), followed by Platformer (18.4%) and Action-Adventure (9.2%) games, with the remainder distributed across seven genres.

实验结果

核心发现包括四个方面：(1) 动作提取管道的有效性：在基准测试中，摇杆位置的平均$R^2$为0.84，按钮帧准确率为0.96，覆盖Xbox、PlayStation等主流控制器家族，证明自动解析的可靠性；(2) 预训练模型的零样本能力：在10个游戏的30个任务上，未微调的模型就能实现非平凡的完成率，2D俯视游戏平均61.5%，2D侧卷游戏平均54.0%，3D游戏平均52.0%，表明模型能够泛化到未见过的游戏；(3) 数据集噪声的鲁棒性：尽管数据存在延迟、视觉干扰（如直播聊天框）、控制器配置差异等噪声，大规模预训练仍然能得到稳健的策略；(4) 预训练的迁移优势：在低数据 regime（30小时）下，从预训练模型微调相比从头训练，在等距roguelike游戏上平均相对改进10%，在3D动作RPG上平均相对改进25%，其中战斗任务达到52%的相对改进，表明预训练学习了可迁移的游戏技能。

In-game rollouts. We show NitroGen performing tasks in diverse 2D and 3D environments. These tasks can take from a few seconds to a few minutes to perform. Some of them include memorization, while others are performed in procedurally generated worlds and require the model to adapt.
Figure 4: In-game rollouts. We show NitroGen performing tasks in diverse 2D and 3D environments. These tasks can take from a few seconds to a few minutes to perform. Some of them include memorization, while others are performed in procedurally generated worlds and require the model to adapt.
Gamepad parsing performance for different controller families. We verify the correctness of our action extraction pipeline by comparing performance across different controller families against ground-truth data. (a) shows joystick R² correlation scores (averaged for both left and right joysticks) with an overall average of 0.84. (b) shows button frame accuracy with an overall average of 0.96.
Figure 5: Gamepad parsing performance for different controller families. We verify the correctness of our action extraction pipeline by comparing performance across different controller families against ground-truth data. (a) shows joystick R² correlation scores (averaged for both left and right joysticks) with an overall average of 0.84. (b) shows button frame accuracy with an overall average of 0.96.
NitroGen 500M pre-training results across different games. We evaluate NitroGen after behavior-cloning pre-training. The model is not fine-tuned for specific games. For each game, we measure the average task completion rate on 3 tasks with 5 rollouts per task. Despite being trained on a very noisy internet dataset, NitroGen is able to perform non-trivial tasks over games with different visual styles (3D, 2D top-down, 2D side-scrolling) and genres (platformer, action-RPG, roguelike, etc.).
Figure 6: NitroGen 500M pre-training results across different games. We evaluate NitroGen after behavior-cloning pre-training. The model is not fine-tuned for specific games. For each game, we measure the average task completion rate on 3 tasks with 5 rollouts per task. Despite being trained on a very noisy internet dataset, NitroGen is able to perform non-trivial tasks over games with different visual styles (3D, 2D top-down, 2D side-scrolling) and genres (platformer, action-RPG, roguelike, etc.).
Post-training experiments: NitroGen pre-training improves downstream agents in unseen environments. We pre-train NitroGen on the dataset described in Section 2.1, holding out one game. We then fine-tune the pre-trained checkpoint on the held-out game and compare the results with a model trained from scratch using the same architecture, data and compute budget. (a) When varying data quantity, task-completion rate scales with dataset size, and fine-tuning achieves on average a 10% relative improvement in task-completion rate. (b) When varying task type in the low-data regime (30h), fine-tuning achieves up to 52% relative improvement in task-completion rate.
Figure 7: Post-training experiments: NitroGen pre-training improves downstream agents in unseen environments. We pre-train NitroGen on the dataset described in Section 2.1, holding out one game. We then fine-tune the pre-trained checkpoint on the held-out game and compare the results with a model trained from scratch using the same architecture, data and compute budget. (a) When varying data quantity, task-completion rate scales with dataset size, and fine-tuning achieves on average a 10% relative improvement in task-completion rate. (b) When varying task type in the low-data regime (30h), fine-tuning achieves up to 52% relative improvement in task-completion rate.
查看结构化数据
任务指标本文基线提升
动作提取-摇杆位置 R²分数 0.84 (所有控制器平均) 不适用 首次实现互联网规模自动标注
动作提取-按钮状态 帧准确率 0.96 (所有控制器平均) 不适用 首次实现互联网规模自动标注
等距roguelike游戏微调 (120h数据) 任务完成率 57.8% 48.1% (从头训练) 20% 相对改进
3D动作RPG微调 (30h数据) - 战斗任务 任务完成率 81.0% 53.0% (从头训练) 52% 相对改进
3D动作RPG微调 (30h数据) - 导航任务 任务完成率 73.3% 60.0% (从头训练) 25% 相对改进

局限与改进

作者承认的局限性包括：(1) NitroGen是一个快速反应的系统1感知模型，无法进行长期规划或遵循语言指令，模型只对短期上下文做出反应；(2) 数据集存在偏差，覆盖了超过1000个游戏但主要集中在动作游戏（34.9%是动作RPG，18.4%是平台游戏），偏向使用游戏手柄的游戏，键盘游戏或需要复杂操作的游戏代表性不足；(3) 通用模拟器通过拦截系统时钟控制仿真时间，虽然适用于大多数游戏，但可能影响某些游戏的物理引擎行为，且不支持实时或异步部署。我观察到的额外局限是：模型只使用单帧上下文，对于需要记忆的任务可能表现不佳；数据集虽然大，但每个游戏的平均小时数相对较少，可能导致对复杂游戏的深度不够。

独立分析的弱点

独立分析的弱点及改进方向：(1) 数据偏差导致泛化受限：当前数据集偏向动作游戏和手柄操作，难以泛化到策略游戏、模拟游戏等需要长期规划的游戏类型。改进方向是扩展数据收集策略，专门收集键盘鼠标操作的游戏、RTS策略游戏等；(2) 缺乏长期记忆和规划能力：模型只使用单帧上下文，无法处理需要记忆的任务（如记住之前的敌人位置或地图探索）。改进方向是引入记忆机制，如循环神经网络或注意力记忆，支持多帧上下文；(3) 不支持语言指令：当前模型完全依赖视觉输入，无法理解自然语言指令。改进方向是加入语言编码器，构建视觉-语言-动作模型，支持任务条件化；(4) 模拟器限制：系统时钟拦截方法可能影响某些游戏的物理，且不支持实时交互。改进方向是开发更底层的接口，如内存读取、网络包拦截等。

未来方向

作者提出的未来工作方向包括：(1) 语言条件化：后训练NitroGen以支持语言指令跟随，增强规划能力；(2) 强化学习：在预训练基础上应用RL微调，提升成功率；(3) 扩展模态：加入音频输入，利用游戏中的声音信息。基于成果可延伸的方向：(1) 多模态扩展：引入目标检测、场景理解等感知模块，构建更丰富的游戏状态表示；(2) 层次化架构：将NitroGen作为低层动作策略，上层使用LLM进行高层规划和推理；(3) 主动学习：让模型主动请求标注或演示，提高数据效率；(4) 跨领域迁移：研究游戏技能如何迁移到机器人等其他具身领域；(5) 元学习：训练模型快速适应新游戏，减少微调数据需求；(6) 安全性和可控性：研究如何确保游戏代理的行为符合预期，避免不良行为。

复现评估

复现性评估：论文声明将开源数据集、模拟器和预训练权重，这是非常好的实践。数据集包含40,000小时视频和解析的动作标签，虽然下载量较大但提供了完整资源。模拟器的Gymnasium API标准化了接口，便于复现实验。模型架构基于公开的GR00T N1和SigLIP 2，实现细节在附录中给出。主要挑战在于算力需求：训练需要大规模GPU资源（原文未给出具体算力，但提到40,000小时视频的大规模训练），但推理阶段相对轻量。动作提取管道的模板匹配和SegFormer模型也开源了，研究者可以在自己的视频上复现。整体而言，开源策略很完整，复现难度主要在于算力资源，而非实现细节不明。

论文图表

figure_1.jpg
figure_1.jpg
figure_2.jpg
figure_2.jpg
figure_3.jpg
figure_3.jpg
figure_4.jpg
figure_4.jpg
figure_5.jpg
figure_5.jpg
figure_6.jpg
figure_6.jpg
figure_7.jpg
figure_7.jpg