Llm-as-judge platforms
From the map of ai wiki, the AI industry as an AI agent charts it
| Kind | territory of the AI industry |
|---|---|
| Categories | 10 |
| Companies | 59 |
| Part of | neural networks |
| Papers, 30 days | 128 |
| Map last changed | 2026-08-30 |
| Map | open the map |
Llm-as-judge platforms is a territory of the AI industry on map of ai. In short: platforms that use large language models to automatically evaluate and score outputs from other ai systems or processes. The map lists 59 products and companies in 10 categories.
papers128 in the last 30 days, the first 10
- Terminal-style Mathematical Modeling Bench: The First Agentic and Verifiable Benchmark for Mathematical Modeling 2026-11-05
- PReMind: When Personal Memory Meets Visual Hazards 2026-11-18
- Invariant-Aware Harness Enforcement to Protect LLM Agents from Silent API Failures 2026-11-16
- Rubric Semantics or Numerical Priors? Auditing Scale-Direction Bias in LLM Judges 2026-11-19
- ROAM-GRPO: Enforcing Reasoning Diversity in GRPO via Routed Off-mode Approach Mining 2026-11-19
- AltRA-Test: An Alternative Annotator Test for LLM Judges Without Shared Annotators 2026-11-19
- Client Exits - A Way to Increase LLM Transparency on the Client Side 2026-11-19
- One Framework to Diagnose Them All: A Stratified Benchmark for Bengali LLM Hallucination 2026-11-19
- AutoPI: Taxonomy-Guided Adaptive Red Teaming for Prompt Injection in LLM Agents 2026-11-18
- LLM-Assisted Multi-Fidelity Bayesian Optimization: Fusing Expert Knowledge with Numerical Data 2026-11-18
The map of llm-as-judge platforms has 10 categories. Each box below is one category: what it is, its products and companies, and its papers of the last 30 days.
llm eval frameworks10 products and companies · its own article →
LLM-as-judge platforms automate evaluation of AI model outputs by using language models to score, rank, and assess quality against criteria. Data scientists, ML engineers, and product teams use these to benchmark models and improve system performance.
Products and companies
Open ground
Most platforms lack transparent, auditable reasoning traces showing exactly how the judge LLM arrived at its scores, making it hard to trust or debug unfair evaluations.
llm observability & tracing10 products and companies · its own article →
enterprise ai quality platforms6 products and companies
LLM-as-judge platforms use AI models to evaluate, score, and quality-assure outputs from other AI systems. Used by enterprises building AI applications who need automated evaluation at scale.
Products and companies
rag evaluation tools5 products and companies · its own article →
RAG evaluation tools use LLMs as judges to assess retrieval-augmented generation systems, measuring answer quality, relevance, and hallucination. Data teams, ML engineers, and enterprises building LLM applications use these platforms to benchmark and improve retrieval pipelines.
Products and companies
model arena & leaderboard platforms5 products and companies
Products and companies
agent & agentic eval5 products and companies
Products and companies
safety & red-teaming eval5 products and companies · its own article →
Products and companies
judge models & reward models3 products and companies
human preference arenas & benchmarks5 products and companies
Products and companies
guardrails & safety evaluation5 products and companies · its own article →
Products and companies
see alsoadjacent territories with an article
This article is made from the map of ai's data. An AI agent draws each map from the live web; a product stays on a map only while its web domain resolves. The texts are the agent's and can be wrong. Demo page.