hmnshudhmn24/llm-benchmarks-capabilities-2020-2026

📊 LLM Benchmarks & Capabilities 2020–2026

The most comprehensive open dataset tracking the evolution of Large Language Models — from GPT-3 to GPT-5.5, Claude Opus 4.7, Gemini 3.5, and beyond.

---

🧭 Overview

This dataset captures the complete LLM landscape from 2020 to 2026 across five dimensions:

• 🤖 113 models from 25+ organizations
• 📈 17 benchmarks tracking capability growth over time
• 💰 Monthly API pricing showing 100x+ cost reductions
• ⚙️ Training compute estimates validating scaling laws
• 🏁 57 capability milestones marking key inflection points

Designed for ML researchers, AI practitioners, policy analysts, and data scientists working on LLM-related problems — trend analysis, capability forecasting, cost-performance modeling, and competitive intelligence.

---

📁 Dataset Files

File	Rows	Description
models_catalog.csv	113	Model metadata: org, release date, params, type, access
benchmark_scores.csv	1,276	Long format: model × benchmark × score
pricing_history.csv	1,187	Monthly API pricing per model (USD per 1M tokens)
compute_estimates.csv	113	Training FLOPs, GPU hours, cost, energy, CO2
capability_milestones.csv	57	Major AI events with significance scores

Total: ~2,750 rows · ~250 KB

---

🏢 Organizations Covered

Closed Frontier OpenAI · Anthropic · Google DeepMind · xAI · Microsoft

Open Weights Meta · DeepSeek · Mistral · Alibaba Qwen · 01.AI · TII

Early Era Chinchilla · PaLM · BLOOM · OPT · GLaM · Switch Transformer

Chinese Frontier WuDao 2.0 · ERNIE 3.0 · HyperCLOVA · YaLM 100B

---

📐 Benchmarks Tracked

Benchmark	Type	Max Score
MMLU	Knowledge	100
MMLU-Pro	Knowledge Hard	100
HumanEval / HumanEval+	Coding	100
MBPP	Coding	100
GSM8K	Math Grade	100
MATH	Math Competition	100
AIME 2024	Math Olympiad	100
GPQA Diamond	Science PhD	100
HellaSwag	Commonsense	100
ARC-Challenge	Reasoning	100
TruthfulQA	Truthfulness	100
BBH (BIG-Bench Hard)	Reasoning Hard	100
SWE-Bench Verified	Agentic Coding	100
LiveCodeBench	Live Coding	100
MMMU	Multimodal	100
Chatbot Arena ELO	Human Eval	1500

---

⚡ Quick Start

import pandas as pd

# Load models catalog
models = pd.read_csv("hf://datasets/hmnshudhmn24/llm-benchmarks-capabilities-2020-2026/models_catalog.csv")

# Load benchmark scores
scores = pd.read_csv("hf://datasets/hmnshudhmn24/llm-benchmarks-capabilities-2020-2026/benchmark_scores.csv")

# Load pricing history
pricing = pd.read_csv("hf://datasets/hmnshudhmn24/llm-benchmarks-capabilities-2020-2026/pricing_history.csv")

# Load compute estimates
compute = pd.read_csv("hf://datasets/hmnshudhmn24/llm-benchmarks-capabilities-2020-2026/compute_estimates.csv")

# Load capability milestones
milestones = pd.read_csv("hf://datasets/hmnshudhmn24/llm-benchmarks-capabilities-2020-2026/capability_milestones.csv")

print(models.head())

---

💡 Suggested Use Cases

1. Benchmark Saturation Forecasting — MMLU went from 32% (2020) to 95% (2025). Predict saturation for GPQA Diamond and SWE-Bench.
2. Price vs Capability Analysis — Plot Arena ELO vs blended API price. Track Pareto-optimal models per quarter.
3. Open vs Closed Model Gap — Measure the capability gap between open weights and closed models across time.
4. Scaling Law Validation — Plot training FLOPs vs benchmark scores. Test empirical scaling exponents.
5. Reasoning Model Premium — Measure score lift of reasoning models vs chat models on math and coding benchmarks.
6. Cost Reduction Trajectory — Track price per Arena ELO point over time. How fast is intelligence-per-dollar growing?
7. Competitive Organization Analysis — Head-to-head benchmark matrix per organization per quarter.
8. Chinese vs Western Capability Race — DeepSeek, Qwen, ERNIE vs OpenAI, Anthropic, Google.