Separate missed issues and false alarms before praising accuracy

A high correct-total fraction can coexist with missing every issue in a small example. Inspect the kinds of error before praising the total. This manual exercise uses invented cards with stipulated labels and predetermined flags. It is not a performance rating of colleagues, a Kiguri analytics report, or an estimate of real issue prevalence.

Fix the reference labels and the proposed flags

Create ten fictional cards, C1 through C10. The exercise stipulates that C1 has a broken reference and the other nine are valid. These labels are supplied for practice, not inferred from actual documents. Keep each identity and its label in the record.

Rule A flags none. Rule B flags C1, C2, and C3. The remaining cards are unflagged. These are fixed toy outputs, not two deployed checkers. Compare both against the same reference labels so a change in the supplied cases cannot quietly explain a different result.

Account for all four outcomes

For each rule, distinguish an actual issue correctly flagged, a valid card incorrectly flagged, an actual issue left unflagged, and a valid card correctly left unflagged. Count every card exactly once.

Rule A has zero caught issues, zero false alarms, one missed issue, and nine correctly unflagged cards. Rule B has one caught issue, two false alarms, zero missed issues, and seven correctly unflagged cards. Both sets of counts total ten. Keep the individual card references behind each cell so the totals can be reconstructed.

Calculate the different denominators

Total correctness counts caught issues plus correctly unflagged valid cards, divided by all ten cards. Rule A gives nine out of ten; Rule B gives eight out of ten. Under that single measure, A looks better.

The fraction of actual issues caught uses the one stipulated issue as its denominator. A gives zero out of one, while B gives one out of one. The fraction of flags that are actual issues uses flagged cards as its denominator. B gives one out of three. A has no flags, so that fraction has no denominator and is undefined; it is not automatically perfect or zero.

Explain the tradeoff without choosing by one total

Rule A's higher correct-total fraction comes entirely from leaving valid cards unflagged; it catches no issue in this fixture. Rule B catches the supplied issue but also asks for attention on two valid cards. Both observations matter to a decision about what the rule should accomplish.

Do not conclude that more flags or fewer flags is always preferable. Keep the cost and intended handling of missed issues and false alarms as an explicit question for the responsible owner. The constructed one-issue/nine-valid composition does not establish how common issues are in real work.

Keep a bounded outcome record

Retain card labels, outputs, four-cell counts, formulas, and undefined results together. Ask another reader to identify which cards explain the difference between nine-out-of-ten correctness and zero-out-of-one issue detection. If they confuse the denominators, revise the summary.

The finish check is an accurate account of these stipulated outputs, including both kinds of error. No people were scored and no actual checker was run. The worksheet supports a clearer question about a future review rule, while leaving real reliability, data quality, and operating policy to appropriate evidence and authority.