Separate rule selection from a withheld check
A candidate can reproduce answers it already copied without demonstrating how it handles a new case. Keep the evidence used to choose it separate from a later check. This manual exercise uses invented identifiers and supplied reference answers in an ordinary shared document. It is not an actual learning-system evaluation, a Kiguri validation feature, or a population-accuracy claim.
Record which references are exposed
Prepare four selection cases: E1 is false, E2 is true, E3 is false, and E4 is true. The rule selector may inspect these answers. Keep two cases separate during selection: H1 has a stipulated reference of true and H2 has false, but their references are withheld from the selector until evaluation.
Write an exposure ledger identifying the cases, when their answers become available, and what the selector can use. The labels selection and withheld matter only if that separation actually occurs. Moving a case into a different column after its answer was already inspected does not make it new evidence.
Freeze an explicit lookup candidate
For this fixture, the candidate simply stores the four seen identifier-answer pairs. It returns the stored answer for a known identifier and unknown for an identifier absent from its table. Unknown is an explicit output, not an omitted row or a default false value.
Record that behavior before opening the withheld references. The candidate has no additional rule for interpreting an identifier. Do not invent such a rule later to make its evaluation look stronger. Its version should remain fixed while the supplied cases are checked.
Compare the seen and withheld results
On E1 through E4, the lookup returns false, true, false, and true. All four match the exposed references. That result is unsurprising because the candidate copied those entries. It does not establish that four independent new cases were handled correctly.
On H1 and H2, the same frozen candidate returns unknown for both identifiers. After revealing their references, retain those unknown predictions alongside true and false. The fixture therefore demonstrates four copied matches and two unresolved outputs. It does not justify counting the two withheld cases as correct or incorrect under an unstated fallback convention.
Keep selection and evaluation decisions distinct
If someone adds H1 and H2 to the lookup table after seeing their answers, label that as a revised candidate using additional exposed information. Retesting it on those same identifiers would inspect the revised table, not recreate the original withheld check.
A different exercise could define a broader rule and new independent cases, but it would need its own declared behavior and evidence. Do not convert this small lookup example into a claim that a particular evaluation method guarantees generalization. The ledger records exposure and outputs, not the representativeness of a sample or the performance of a real system.
Check the account of what was established
Ask another reviewer which results came from stored answers and which came from previously unseen identifiers. They should find the candidate version, its unknown-output rule, and the timing of reference exposure without reconstructing the conversation.
Preserve all six cases in the record. A summary saying the candidate passed its checks loses the crucial distinction between copied references and unresolved new cases. The useful outcome is a transparent selection-and-check history, with unknowns intact and no stronger capability claimed than the candidate actually demonstrated in the fictional fixture.
