One of the major hurdles I have struggled to overcome is the lack of exploration for engine determinism and its ramifications for building up a quality platform for a long-term game. Often the topic is glossed over or even outright ignored, with no discussion of seeding random values, writing multithreading such that order of operations is respected and not subject to system whim, and so forth.
When reproducing defects in game mechanics, the problem becomes exponentially harder than in a standard software system, as the effects of nondeterminism rapidly compound with each other due to the highly dynamic nature of games. In brief, a degenerate case may appear only once for any given random value and only when three given subsystems run in a specific order.
Reconstructing these conditions from a user experience report, or even from a debug log, becomes a question of time and effort costs: what is the estimated likelihood of being able to track down something like this? Rampant bugs may be more easily reproducible in shorter time, but their resolution may be nontrivial if their existence is based on some chance ordering of subsystem execution, for example.
Bringing engine determinism to the forefront of considerations in design, then, may provide a solid foundation for reproducing and eliminating bugs. All that is necessary is a state capture, and from that point the issue can either be ruled a hardware or OS problem (drivers, render pipeline, etc) or can be tracked to the specific combination of conditions necessary to cause the failure in the game logic by recreating the state.
In some cases, determinism may not be fully possible. High-performance games may rely on nondeterminism as an optimization mechanism, allowing state to be eventually consistent but with varying arrangements in execution path as processor load allows. Even so, keeping such reliances to a minimum reduces the potential surface area for bugs that cannot be reproduced simply by state recreation.
To some extent, this also allows for ideas such as unit testing and replay testing for the game engine, ensuring that established behaviors are not subsequently broken by the introduction of new features.
In a future post I’ll explore some options for tracking and enforcing determinism. Some engines/frameworks offer support built in, while others may require special measures to isolate game logic in a deterministic form.