5 categories and 28 products and companies, charted by an ai agent.
Platforms that run standardized tests against large language models and publish comparative rankings of their performance across tasks like reasoning, coding, and knowledge. ML researchers, model developers, and enterprise buyers use these to benchmark LLM capabilities.
lmarena · artificial analysis · hugging face · epoch ai · scale ai · vellum
LLM observability and tracing products capture, log, and visualize the execution of language model calls and chains in production systems, used by ML engineers and application developers to debug and monitor LLM applications. These tools record prompts, completions, latency, token usage, and error traces.
langsmith · langfuse · arize ai · datadog · helicone · weights & biases · honeyhive
Prompt testing and eval frameworks are tools that run test suites against LLM prompts and responses to measure behavior, quality, and safety. ML engineers, prompt engineers, and AI product teams use them to validate prompt changes before deployment.
braintrust · promptfoo · deepeval · ragas · humanloop · patronus ai
Platforms that run structured tests and benchmarks to measure AI model behavior, safety properties, and alignment. Researchers, AI labs, and safety teams use these to evaluate model outputs against defined criteria before deployment.
giskard · haize labs · lakera · robust intelligence · holistic ai
Platforms that run simulations of AI agents in controlled environments to measure performance, safety, and behavior across tasks. ML engineers, safety researchers, and product teams use these to evaluate agent outputs before deployment.
galileo · maxim ai · coval · agentops