# Methodology v1.0

A model is SOTA in a category on a date when it leads the approved primary evidence beyond the margin of error.

Crown changes need two consecutive snapshots of a primary source, or two independent primary sources.
When primary sources conflict, the verdict is low-confidence co-SOTA.
Badges never say NOT SOTA; they show the best rank instead.
Self-reported vendor numbers are trust 0.5 and cannot hold a crown by themselves.
SWE-bench Verified and METR time horizons are display-only.

- livebench kept LiveBench/version LiveBench-2024-11-25 (1 versions seen).
- Artificial Analysis was skipped because ARTIFICIAL_ANALYSIS_API_KEY is unset. Intelligence, coding, agentic, and speech-arena scores are not used. Text-to-speech has no crown until that key is set. Raw scores are never written to the public catalog.
- Artificial Analysis was skipped because ARTIFICIAL_ANALYSIS_API_KEY is unset.