Fix the judge, by hand

Twelve items, a mock judge, and the score it earns

Each card is one support item: the human gold label on top, the judge's actual raw verdict below. Click a card to swap that one verdict for an idealized answer that agrees with the human, and watch the confusion matrix and all four metrics recompute.

Click an item to toggle between the judge's actual verdict (as printed by the mock judge) and a fixed verdict that agrees with the human gold. Fixing Q05 removes the false positive; fixing Q06 and Q09 removes the two false negatives; fixing Q02 or Q07 pulls an abstained item back into the scorable set.

Scorable set

Judge vs human

Scores

Accuracy0.00
Precision0.00
Recall0.00
F10.00