Source note2 min read
The Human Creativity Benchmark
Source
arXiv
arxiv.org/abs/2606.30561v1
Source
- Source name: arXiv
- Source kind: paper
- Authors / organization: Aspen Hopkins, Allison Nulty, Alexandria Minetti, Anoop Pakki, Angad Singh
- Venue: arXiv: cs.AI
- Published: 2026-06-29
- Collected: 2026-06-30
- URL: http://arxiv.org/abs/2606.30561v1
- PDF: https://arxiv.org/pdf/2606.30561v1
- DOI: UNKNOWN
Plain-Language Brief
The authors propose a new benchmark for evaluating creative AI called the Human Creativity Benchmark (HCB). This benchmark separates two signals of creativity, convergence and divergence, by collecting preferences, ratings, and qualitative feedback from domain professionals. The study finds that convergence focuses on verifiable dimensions like technical correctness and visual hierarchy, while divergence focuses on taste-driven dimensions like aesthetic direction and conceptual risk. No model performs well across all phases.
What the Document Says
- The Human Creativity Benchmark (HCB) is proposed for evaluating creative AI.
- HCB collects pairwise preferences, scalar ratings on prompt adherence, usability, and visual appeal, and qualitative rationale from domain professionals.
- 15,000 professional judgments were collected across five creative domains and three workflow phases (ideation, mockup, refinement).
- Convergence concentrates on verifiable dimensions like technical correctness and visual hierarchy.
- Divergence concentrates on taste-driven dimensions like aesthetic direction and conceptual risk.
- No model excels uniformly across all phases.
- Collapsing these signals into a single quality metric discards the most actionable information.
Key Concepts Mentioned
- Human Creativity Benchmark
- creative AI
- convergence
- divergence
- professional judgments
- creative domains
- workflow phases
- technical correctness
- visual hierarchy
- aesthetic direction
- conceptual risk
Claims Made by the Document
- The Human Creativity Benchmark is useful for evaluating creative AI.
- Convergence and divergence are two distinct signals of creativity.
Evidence Used
- Collecting pairwise preferences, scalar ratings, and qualitative rationale from domain professionals.
- Findings from the study across different domains and phases
Methods / Approach
UNKNOWN
Limitations Stated by the Authors
UNKNOWN
Limitations Noted by the Agent
Agent interpretation — not from the source.
UNKNOWN
Open Questions
UNKNOWN
Notes for Future Q&A
May include interpretation. Labeled where it does.
UNKNOWN
Raw Abstract / Excerpt
Modern AI evaluation frameworks treat evaluator disagreement as noise to be resolved. In creative domains, professional disagreement reflects genuine differences in taste, not measurement error. We argue that evaluating creative AI requires preserving two distinct signals: convergence, where professionals align around shared best practices, and divergence, where individual taste legitimately varies. We present the Human Creativity Benchmark (HCB), a benchmark that operationalizes this separation by collecting pairwise preferences, scalar ratings on prompt adherence, usability, and visual appeal, and qualitative rationale from domain professionals. Across 15,000 professional judgments spanning five creative domains and three workflow phases (ideation, mockup, refinement), we find that convergence concentrates on verifiable dimensions like technical correctness and visual hierarchy, while divergence concentrates on taste-driven dimensions like aesthetic direction and conceptual risk. No model excels uniformly across all phases. Collapsing these signals into a single quality metric discards the most actionable information: where models must be correct versus where they should remain steerable.
