Every enemy in Ruinveil has a behavior tree. That statement is not unusual; behavior trees are standard architecture in game AI and have been for fifteen years. What is less standard is that our behavior trees are not fixed per enemy type. They are assembled at the start of each run from a pool of behavioral traits, composited into a tree structure that can differ between runs even for what looks like the same enemy archetype.
This post explains why we made that choice and how the trait composition system works in practice.
The Problem with Fixed Scripts
The traditional approach to enemy AI in action games is to define a behavior script per enemy type. The Crawler does this: patrol, detect, chase, attack, retreat at low health. The Sentinel does this other thing: hold position, fire, reposition, summon. Each enemy has a well-defined behavioral identity, and players learn it through repetition.
This works. It produces predictable, readable enemies that players can develop mastery over. In a game that respects mastery, that is usually the correct design. But in a roguelite where the central promise is that every run is a different problem, fixed scripts create a ceiling on that promise. Once you know how an enemy type behaves, you know it across every run. The dungeon layout might change, but the enemy you meet in room seven behaves exactly as the same enemy type behaved in room seven of your previous four runs.
We are not saying fixed scripts are a bad design choice in general. For games where mastery is the primary reward, they are often the right call. But for Ruinveil, where we want each run to present genuinely different problems, fixed scripts undermine the variety we are trying to produce at the dungeon level.
Behavioral Traits as Building Blocks
Our behavior tree architecture starts from a library of behavioral traits. A trait is a self-contained behavioral module that defines a specific decision pattern: how an enemy evaluates threat priority, when it decides to reposition, whether it coordinates with nearby allies, how it responds to taking damage, what its aggression decay curve looks like over the course of an engagement.
Each trait is authored as a subtree that can be grafted into a parent behavior tree at designated attachment points. The trait library currently has around forty defined traits. Some examples to make this concrete:
Flanking tendency: when a valid flanking position exists and the trait's aggression threshold is met, the enemy attempts to route toward the player's side or rear rather than approaching directly. The strength of this tendency is parameterized; a high flanking value produces enemies that actively prioritize angle exploitation, while a low value makes it an occasional behavior.
Retreat-and-return: at a configurable health threshold, the enemy breaks off engagement and attempts to reach a defensible position, then re-engages after a delay. This differs from a standard retreat because the enemy reassesses and returns rather than fleeing indefinitely.
Threat broadcasting: the enemy signals its threat status to nearby allies within a radius. Allies that receive this signal can adjust their own behavior in response. The broadcasting radius and signal type are parameters of the trait.
Assembling a Behavior Tree at Run Start
When Ruinveil initializes a run, the enemy ecology system determines what enemy composition the dungeon will have and what ecological roles each enemy fills. After that assignment, the behavior construction pass runs: for each enemy instance, a set of traits is drawn from the pool according to the enemy's ecological role and the run's entropy seed.
A harasser-role enemy might receive a high flanking trait, a retreat-and-return trait, and a low-aggression opening behavior. A sentinel-role enemy might receive a threat broadcasting trait, a high-aggression opening, and no flanking tendency. The draw is weighted rather than purely random: the ecological role establishes a probability distribution over the trait pool, and the seed determines the specific draw from that distribution.
The assembled traits are grafted into a base skeleton tree. The skeleton defines the fundamental decision structure: perceive, evaluate, select action, execute, repeat. The traits modify and extend specific decision branches within that skeleton. The result is a behavior tree that is unique to this enemy instance in this run, but structurally coherent because it derives from a well-formed skeleton with well-formed attachments.
What the System Gets Right
When it works well, the trait composition produces enemies that behave in ways that feel unexpected without feeling arbitrary. A player who has learned that a particular enemy archetype tends to approach directly encounters one that breaks off at moderate health and repositions, and has to revise their mental model mid-engagement. The revised behavior was not scripted; it emerged from a trait combination that this particular enemy drew. The player cannot have memorized it because it did not exist in the same form before this run.
This is the behavioral equivalent of what the dungeon grammar does at the spatial level: generating encounters that feel intentional even though they were not hand-authored. A run where the enemies happen to have a high coordination of threat-broadcasting plus flanking tendency produces fights that feel qualitatively different from a run where those traits are absent, and the player registers that difference even without articulating its source.
Where the System Breaks Down
The place where the system breaks down is in trait interaction edge cases. Some trait combinations produce behaviors that are incoherent: an enemy with both a high flanking tendency and a threat-broadcasting trait that expects allies to converge on it can create a situation where the enemy is actively moving away from its allies while simultaneously signaling them to approach. The resulting behavior is neither intentional flanking nor effective coordination.
We have a compatibility constraint layer that prevents clearly incompatible trait pairings, but the space of subtle incompatibilities is larger than we have fully mapped. When playtesters describe an enemy as "acting weird" without being able to articulate why, the root cause is usually a trait combination that passed our compatibility rules but produces a behavioral incoherence we did not anticipate. Finding and closing those cases is ongoing work.
Performance Considerations
Behavior trees have known performance characteristics in game engines, and assembling them at runtime adds overhead. We run the tree assembly pass during dungeon generation, before the player enters the level, so the per-frame tick cost during gameplay is the same as a static tree. The assembly itself costs roughly 2ms per enemy instance for a tree with three to five traits; a fully populated dungeon with twenty-eight active enemy instances takes under 60ms total.
The tick evaluation at runtime is where performance matters most. Our trees are intentionally shallow: we avoid deep nesting in favor of wider parallel evaluation, with a maximum tree depth of seven nodes in any assembled configuration. This keeps tick cost predictable and avoids cascading subtree evaluations that can become expensive under specific game states.
What We Have Not Solved Yet
The largest open question in our behavior system is adaptation across a single run. Our trees currently evaluate based only on present game state. Enemies do not retain information about past player encounters within the same run, which means a player who exploits an enemy's flanking tendency in room four will find the same tendency exploitable in room twelve. There is no within-run adaptation, only variation between runs.
Adding within-run learning is on our design list but it requires persistent state per enemy instance and careful thought about whether adaptation improves the player experience or tips into frustration. We are not ready to commit to an approach yet, and we want playtest data to tell us whether the current system is already close enough to what we need.