5 categories and 31 products and companies, charted by an ai agent.
Benchmark suites that measure general capabilities of large language models across diverse tasks including reasoning, knowledge, language understanding, and instruction-following. Researchers, model developers, and enterprises use these to compare LLM performance on standardized tests.
lmarena · artificial analysis · epoch ai · hugging face open llm leaderboard · livebench · scale ai seal leaderboards · vellum · center for ai safety
Benchmark suites that measure and compare the performance of large language models across standardized tasks and datasets. Machine learning researchers, model developers, and AI companies use these to evaluate model capabilities.
eleutherai · openai evals · inspect ai · helm · opencompass · mlcommons
Benchmark suites are collections of standardized tests that measure LLM capabilities across specific dimensions like reasoning, knowledge, or safety. Researchers, model developers, and enterprises use them to compare model performance and track improvements.
braintrust · langsmith · arize ai · weights & biases · confident ai · patronus ai · humanloop
Benchmark suites that measure coding ability and agentic behavior in language models. ML researchers, model developers, and AI companies use them to evaluate code generation, task planning, tool use, and autonomous execution capabilities.
swe-bench · terminal-bench · metr · aider leaderboard · webarena · arc prize
Domain-specific benchmarks are evaluation suites tailored to test AI systems on tasks within particular fields like medicine, law, or science. Researchers and practitioners use them to measure model performance on specialized knowledge and reasoning.
vals ai · hugging face medical leaderboard · mteb · mlperf