Calibrating Self-Knowledge Through the Guide
Question. Does working with a guide make an operator's sense of how good they are match how good they actually are, where practising alone does not?
Analogy. Your own writing is the hardest thing to proofread: you read what you meant, not what is on the page, so an error you have read past a dozen times can stay invisible while a fresh reader catches it in a second, and re-reading your own draft is no substitute for the second pair of eyes. The study asks whether a guide is that second reader for an operator's sense of their own skill, or whether the gap between what they think they know and what they actually know is fixed, and even an outside read cannot close it.
What's at stake. A person paired with autonomy is only safe if they know what they actually know: an operator who overrates their grasp of a method acts past their competence, and one who underrates it defers when they should not. The proposal's human-autonomy-teaming claim rests on the operator's self-knowledge being accurate, and the IDEaS Cognition-and-Trust challenge names this exact property, real-time dynamic calibration, as its target. But accurate self-assessment is the thing people are reliably worst at: the least skilled overestimate most and cannot see the gap without outside feedback. Missionloops bets the guide is that outside feedback, a second person who has watched the operator's actual reasoning and can correct an inflated or deflated self-read. If the bet is wrong, the self-assessment half of the mastery model is measuring confidence, not knowledge, and the platform cannot safely tell an operator what they are ready for.
The two answers it decides between. Either metacognitive calibration is trainable through the guide, so guided operators' self-assessments converge toward their demonstrated proficiency over cycles while solo operators stay miscalibrated; or self-assessment is a stable, separable faculty that external feedback does not move (the Dunning-Kruger dissociation), so guided and solo operators are equally miscalibrated and the convergence the pilot expects does not appear. The guided-versus-solo convergence trajectory tells them apart.
What a null result would mean. If guided operators' self-assessment converges with demonstrated proficiency no better than solo operators', the guide-as-calibration claim fails and the dual-path mastery model's self-assessment leg is not a usable proficiency signal: the model would have to lean entirely on behavioural and external measures. That is a finding about whether metacognitive calibration is trainable, not a sign the platform was built wrong, and it directly bounds how much weight the proficiency histogram can place on self-report.
Why this matters to defence. Calibrated self-knowledge is the precondition for safe human-autonomy teaming (DRDC Objective 3, and the IDEaS Challenge's named real-time-dynamic-calibration criterion), and an operator who cannot tell what they know is exactly the soft target cognitive warfare exploits (DRDC Objective 6). It changes a concrete decision: whether self-report can be trusted as a calibration signal in deployed human-autonomy systems, or whether a human second party is required to keep it honest, and whether the Component-2 mastery instrument is valid enough to scale.
How we would run it. Take about fifty operators. Half work with a guide, half work solo; everyone runs at least two hundred missions between them. Two numbers per operator: how good they think they are, from a short self-test built on a standard skills checklist; and how good they actually are, from a wargame on real problems and from a trained observer who watched real missions. We lean on those last two as the real-skill yardstick on purpose, because the platform's own behind-the-scenes signal is expected to be weak and to get weaker as good operators write less, which the study measures rather than assumes. The whole study is one comparison: does working with a guide pull a person's self-rating closer to their real skill, faster and further, than working solo? Two honesty checks: more than one observer rates the same missions and we confirm they agree, so the yardstick is not one person's opinion; and the platform-signal-versus-self-rating comparison, the pilot's official number, is reported only as a secondary, since we already expect that signal to be weak.
Earliest start. Stage 7: the study needs the assessment protocol, the wargame, and the researcher role operational, and runs on the Component-2 pilot timeline.