Audit layer

Every score has an evidence trail.

CreativeBench is designed as event-sourced records: prompts, model responses, judge runs, pairwise comparisons, revision deltas, evaluator reputation, and release metadata remain auditable and reproducible.

`prompt_spec`

prompt_id, release, track, text, genre, constraints, rubric_version, provenance, license, contamination_status

`candidate_response`

response_id, prompt_id, model_id, decoding, text_hash, token_count

`judge_run`

judge_run_id, judge_id, judge_model, rubric_version, dimension scores, confidence, critique_hash

`pairwise_comparison`

comparison_id, response_a, response_b, winner, margin, position_swapped, judge_id

`revision_delta`

draft_hash, critique_hash, revised_hash, revision_lift, regression_risk, preserved_voice

`evaluator_profile`

trust_score, specialist_badges, calibration agreement, conflicts, reciprocity tier

Aggregation

From raw events to public rank.

Compute per-dimension rubric means with judge reliability weights.
Fit Bradley-Terry / Elo / Glicko from pairwise outcomes.
Report bootstrap confidence intervals over prompts and judge cohorts.
Publish category ranks, not just one total rank.
Show diagnostic badges: high originality, strong dialogue, verbosity risk, longform drift, high uncertainty.

Reproducibility contract

Model-output hashes, critique hashes, scoring-script commits, release names, track labels, prompt contamination state, and judge versions stay attached to the public score receipt.