Rubric score
Calibrated 0–100 dimension scores with concrete text evidence.
Method v0.1
Rubrics explain. Pairwise preferences rank. Diagnostics expose failure.
Primary public rank is a blinded pairwise preference score with confidence intervals. Rubrics remain visible as the explanation layer, while diagnostics show verbosity, repetition, slop, uncertainty, and drift.
Pairwise preference
Bradley-Terry / Elo / Glicko-style ranking from blinded comparisons with position swaps.
Diagnostics
Non-quality metrics: slop, repetition, vocabulary control, length, uncertainty, judge disagreement, and drift.
Rubric dimensions
Visible sub-scores.
Every score is explainable and tied to a creative-writing failure or success mode.
Constraint adherence
Narrative coherence
Character consistency and motive believability
Dialogue naturalness and subtext
Style / voice control
Imagery and sentence-level craft
Originality and cliché avoidance
Emotional impact
Pacing and escalation
Long-range continuity
Revision quality
Safety / legal / policy compliance
Negative criteria
Catch common LLM writing failures.
Purple proseGeneric uplift / forced positivityTell-don't-showOverwritten metaphorCliché twistsMeandering setupWeak or samey dialogueUnearned character transformationTonal driftSlop phrases and synthetic cadenceVerbosity inflation
Bias controls
Judge the work, not the wrapper.
- Blind model identity
- Position swapping for pairwise comparisons
- Length-control regression
- Markdown and format controls
- Judge ensemble rotation
- Gold calibration anchors
- Duplicate / near-duplicate prompts for consistency
- Bootstrap confidence intervals
- Prompt contamination labels
Revision is first-class
CreativeBench records initial draft → panel critique → standard editorial brief or own-critique revision → original-vs-revision judgment → revision lift and regression risk. It tracks whether the rewrite improves craft without breaking constraints, flattening voice, adding slop, or losing emotional core.