9 categories and 45 products and companies, charted by an ai agent.
General LLM benchmarks measure language model capabilities across diverse tasks like reasoning, knowledge, and language understanding. ML researchers, model developers, and enterprises use these benchmarks to evaluate and compare large language models.
lmarena · artificial analysis · epoch ai · scale ai seal leaderboards · hugging face · livebench · stanford helm · vals ai · mmlu · hugging face open llm leaderboard
Frontier reasoning benchmarks evaluate large language models on complex reasoning tasks including mathematics, logic, and multi-step problem-solving that require novel approaches beyond pattern matching. Researchers, model developers, and AI safety teams use these benchmarks to measure reasoning capabilities of frontier-class models.
arc prize · humanity's last exam · center for ai safety · frontiermath
Benchmark datasets that measure how well AI agents perform on tasks requiring planning, tool use, and multi-step reasoning. Researchers, ML engineers, and companies building autonomous systems use these to evaluate agent capabilities.
swe-bench · terminal-bench · metr · sierra tau-bench · livecodebench · aider leaderboards
Benchmark datasets and evaluation frameworks designed to test AI systems for harmful outputs, jailbreaks, and unsafe behaviors. Used by AI labs, safety researchers, and red-teamers to identify failure modes before deployment.
uk ai security institute · mlcommons ailuminate · apollo research · gray swan
Hardware performance benchmarks measure computational speed, memory throughput, and power efficiency of processors, GPUs, and accelerators. ML researchers, systems engineers, and hardware vendors use these benchmarks to evaluate training and inference performance on their platforms.
mlperf · inferencemax · geekbench ai
Benchmark datasets are collections of standardized test data used to evaluate machine learning models and AI systems. ML researchers, engineers, and organizations use them to measure model performance across consistent metrics.
braintrust · langsmith · patronus ai · arize · confident ai · weights & biases
Benchmark datasets that measure performance of machine learning models across standard tasks and metrics. Researchers, ML engineers, and product teams use these to evaluate model capabilities before deployment.
mercor · surge ai · openai gdpval · turing · labelbox · invisible technologies
Vision and multimodal benchmarks are standardized test datasets that evaluate how well machine learning models perform on tasks involving images, video, text, and other modalities combined. Researchers, model developers, and ML engineers use these benchmarks to measure performance on object detection, visual reasoning, image captioning, visual question answering, and other cross-modal tasks.
imagenet · coco · mmmu · mlcommons
Agent benchmarks evaluate the performance of autonomous agents on tasks requiring planning, tool use, and decision-making. Machine learning researchers, AI companies, and organizations testing AI systems use these benchmarks to measure agent capabilities.
gaia · webarena