Writing a rubric a model cannot game
Specific beats clever. Four habits for rubrics that keep working after the model gets better.
T
Tomás Reyes
7 min
A rubric is a prompt that grades another prompt. Like any prompt, it improves when it is concrete.
Name the failure
Grade for one failure at a time: invented facts, missing citations, wrong tone. Combined scores hide which one moved.
Give two examples
One that passes and one that fails, both close to the line.
Pin the grader
Record the grader model and version with every score, so a changed score can be explained.
