Recursive AI

Automated game testing with scenarios

All onegame recipes

Write game tests as YAML scenarios that click, press keys, send commands and step frames, then check state, UI, sounds and pictures, and run them headless in CI with onegame test.

A scenario is a test written as text. onegame test runs every file in scenarios/ in a headless game: no window, no display and no GPU needed. This one drives the apple tree from Getting started:

YAML
name: pick-apples
description: Apples grow, picking takes them, and picking more than there are is refused.
seed: 7
steps:
  - expect:
      ui: { id: apples, text: "0 on the tree, 0 picked" }
  - click: pick              # nothing to pick yet: refused
  - step: 300
  - click: pick
  - step: 1
  - command: { entity: Tree, name: Pick, args: [99] }   # more than there are: refused
  - step: 1
expect:
  - state: { entity: Tree, property: Picked, equals: 1 }
  - state: { entity: Tree, property: Apples, equals: 9 }
  - ui: { id: apples, text: "9 on the tree, 1 picked" }
Terminal
onegame test
Text
PASS pick-apples (36 ms, tick 302, hash 9a447aae324c764b)
1 passed, 0 failed

A failing test

With equals: 0 on the Apples line instead, the same run reports the line, the expected value and the actual one, and exits 1:

Text
FAIL pick-apples (37 ms, tick 302, hash 9a447aae324c764b)
  expect state (line 15): Tree.Apples is 9, expected 0
    expected: 0
    actual:   9
0 passed, 1 failed

onegame test --json prints the same as one JSON object, with every step and expectation and its line, for CI to read. The exit code is 0 when everything passed and 1 when a check failed.

What a scenario can do

The top level takes name, description, seed, window (640x360), fixture and snapshot, then steps and expect. Unknown keys are errors with a line number.

Steps
click: pickan element by id; also {selector: "Button"} or {x: 10, y: 20}
step: 300run frames
command: { entity: Tree, name: Pick, args: [2] }send a command as a player would
set: { entity: Tree, property: Apples, value: 40 }write one scalar of state directly
press: Tab, type: "text"keys and typed text
nav: down, nav: activatea controller: up down left right next previous activate cancel, or a button by name
fixture: name, snapshot: file, reset: 77replace the world mid-scenario
expect: ...check something between steps
Expectations
state: { entity: Tree, property: Apples, equals: 9 }a synced property; also min, max; paths like Stock[1] or Stock.count
ui: { id: apples, text: "..." }an element’s text; also selector with count, contains, visible, enabled, focused, pressed
sound: { name: coin, count: 1, frames: 5 }the game played a clip, without a sound device
tick: 3the world’s tick
capture: { reference: refs/main.png, tolerance: 2, max-diff: 0.001 }a picture of the screen against a reference: see Screenshot tests

entity: is a type name (the first entity of that type), a numeric id, or Type#n for the nth. A step that fails stops the scenario; the expectations under expect are checked once every step has run.

Running a subset

Terminal
onegame test --scenario scenarios/pick-apples.yaml    # one file
onegame test --no-unit                                # scenarios only, not the C# test projects

onegame test also runs any C# test project in the game’s folder whose name contains “Test”. See Test a game from C#.

Good to know

  • Scenarios run against the real game: the same host, screens and input path as a player, through the same operations live control uses.
  • A scenario with a seed is repeatable to the bit: the hash in the result is the same on every run.
  • Scenarios run in one process. To test a server and several clients over a real network, a cluster scenario adds node:, wait, agree and drift: see Client/server sync with a dedicated server.
  • onegame web test runs scenarios in headless Chrome, but the web export needs an engine checkout at 1.1.2.

Every key is listed in Driving a game.

A longer guide with replays, screenshot diffs and CI: Automated game testing.