Recap · 2024-09
AI leaderboard changes September 2024
As of Sep 27, 2024, 5 confirmed crown changes are recorded for September 2024.
Crowns
Confirmed crown changes
- Sep 27, 2024. Reasoning: O1 Preview and O1 Mini are co-SOTA. Crown change confirmed by two consecutive snapshots or two independent primary sources. O1 PreviewO1 Mini · Reasoning
- Sep 27, 2024. General LLM: O1 Preview takes the crown. Crown change confirmed by two consecutive snapshots or two independent primary sources. O1 Preview · General LLM
- Sep 24, 2024. Math: Gemini 1.5 Pro 002 takes the crown. First holder recorded for this category and slice. Gemini 1.5 Pro 002 · Math
- Sep 17, 2024. Reasoning: Llama 3.1 405B Instruct Bf16 and Llama 3.1 405B Instruct Fp8 are co-SOTA. Crown change confirmed by two consecutive snapshots or two independent primary sources. Llama 3.1 405B Instruct Bf16Llama 3.1 405B Instruct Fp8 · Reasoning
- Sep 17, 2024. General LLM: Llama 3.1 405B Instruct Fp8 and Llama 3.1 405B Instruct Bf16 are co-SOTA. Crown change confirmed by two consecutive snapshots or two independent primary sources. Llama 3.1 405B Instruct Fp8Llama 3.1 405B Instruct Bf16 · General LLM
Top 3
Models that entered the top 3
- Gemini 1.5 Pro 002 at #1 in Math
- O1 Mini at #1 in Reasoning
- O1 Preview at #1 in General LLM
- O1 Preview at #1 in Reasoning
Benchmarks
Benchmarks added or saturated
Benchmark additions for this month are still empty.