How to assess whether your scenarios are training the right cognitive ability.
You've just finished your scenario design. The character feels real, the situation and options are plausible. You're happy with it. Are you also completely sure about what you're actually having people do? Before you ask if a scenario is well-made, it helps to ask if it's actually practicing the right thing. Are you prompting recognition or judgment?
The gap between recognition and judgment
The concept of transfer-appropriate processing (Morris, Bransford & Franks, 1977) offers a useful perspective here: learning success transfers to practice when the cognitive processes during training match those required on the job. If what is practiced fundamentally differs from the real-world demands, transfer is unlikely.
Trade-off tasks require, on the other hand, what Paris, Lipson, and Wixon (1983) termed conditional knowledge: knowing when and how to act—not just what the right action looks like in the abstract. One cannot develop conditional knowledge without actively practicing judgment itself.
A scenario can only seem like you're training judgment when in reality it demands nothing more than pure recognition. The difference lies in the cognitive Decision point claim, not in how realistic the situation feels.
It's also worth separating two things that are easily conflated:
These are two independent variables. You can have a decision at a recognition level, which is followed by a purely intrinsic consequence. Or a true deliberation task, where a narrator then passes judgment. Both dimensions are important, but both can fail independently.
How that looks in practice
To make that tangible, let's look at two decision points from one of my own interactive stories. Speak or Sink is an interactive story about confident communication in the workplace, designed for young women navigating workplace dynamics and social conditioning. The main character is Lena, a recent graduate. In Scene One, her colleague Jan constantly interrupts her work.
A recognition version of this situation would name the correct answer – „self-assured communication“ is the theme of the story. The incorrect answers would be recognizable solely by their type, without knowing anything else about the relationship or workplace. And the feedback would say something like: „Wrong. In this situation, you should address the problem directly“ – thus replacing a consequence with instruction.
In the original decision point, none of the three options appears obviously wrong. While users might have a general preference, they cannot reliably evaluate the options without analyzing the specific situation closely. If the user chooses „Ask the manager to speak with him,“ Daniel replies, „You don't need to come to me with something like this, Lena.“ A concrete, socially realistic consequence without an explicit principle or narrative judgment. The situation does the work.
Christy Tucker makes a related distinction between Categorization questions and action questions. A categorization question asks people to identify or classify – that's closer to the Level 1 example above. An action question asks them to decide what to do next within the concrete situation. Your point is that options must remain tied to the concrete characters and details, rather than abstracting to the general process.
The „Talk to Jan“ option from Scene 1 still lies on the abstract side of this boundary: the user doesn't know what talking to Jan looks like until the consequence reveals Lena's actual words. Here, the abstraction is supported by strong consequence design – the situation delivers what the option holds back. Shifting this concreteness into the option itself would push the decision even further towards true conditional knowledge.
Christy also emphasizes that classification questions are perfectly valid as practice or test questions. So, it's completely okay to use abstract options. The problem only arises when recognition is mistaken for a deliberation training – or when recognition remains the sole cognitive task in a course that claims to build behavioral competency.
Three dimensions for evaluating decision points
The distinction between recognition and deliberation appears differently depending on the perspective. To turn this into a diagnostic tool, I have divided the whole into three dimensions. In doing so, I use concepts from simulation literature and Ruth Clark's work on scenario feedback – whereby the framework and application to individual decision points (instead of entire scenarios) are my own approach:
I have these three dimensions in a five-stage matrix for scenario decisions merged – from pure knowledge tests in a scenario guise (Level 1) to the actual training of conditional knowledge (Level 5). It is a pragmatic tool for practice, not an empirically validated framework.
Applying the dimensions to a real-world example
In Scene 4, „Manage Your Emotions,“ Lena is publicly criticized in a team meeting. This scene moves between stages 3 and 4 across the three dimensions.
The decision demand is approaching level 4. The options signal a specific strategic move but remain somewhat abstract – the learners choose a strategy, not a concrete, situational action. The exact words only appear on the consequence screen. Abstract options become a problem, particularly when they don't lead to noticeable consequences.
Situational fidelity is between level 3 and 4. The setup is specific and emotionally grounded. However, Daniel's role at the decision point is not explicitly recalled. If the characters' relationships are not precisely remembered, information that would sharpen the judgment is missing. A short line directly before the decision closes this gap.
The authenticity of the consequence is at level 4 to 5. The situation provides the judgment entirely on its own – Steve's comment, Daniel's pause, Tom's interruption. No narrator explains the principle behind it. The learners read the outcome exactly as they would read the dynamics in a real meeting room.
The Swap Test
In my post about Learning with characters Did I introduce the exchange test: Could you replace the character or situation without changing what people need to learn? If so, the scenario is probably just decoration. A modified version of this test also applies here.
Could you replace the decision point with a classic question about the underlying principle, and would you lose anything essential? If so, does the scenario perform a pure knowledge test and not a deliberation training?.
The same principle applies to the options: Could you swap out the characters and setting without rewriting the answer choices? If the options would still work for the same theme in a completely different scenario – with different people and a different context – then they're probably not doing any real situational work.
An effective scenario design not only looks like real life. It also demands what real life demands in crucial moments: genuine deliberation.
Sources: Clark, R. C. (2012). Scenario-based e-Learning (Clark, 2012); Transfer appropriate processing (Morris, Bransford & Franks, 1977); Conditional knowledge (Paris, Lipson & Wixon, 1983); Categorization and action questions (Tucker, 2024 – online).
Edited with Claude Sonnet 4.6 (Anthropic). The ideas are mine.