11 categories and 70 products and companies, charted by an ai agent.
Alignment research labs focus on ensuring AI systems behave safely, reliably, and in accordance with human values as they grow more capable. They are used by governments, frontier AI developers, and academic institutions seeking to reduce existential and societal risks from advanced AI.
anthropic · openai safety · google deepmind safety · machine intelligence research institute · arc (alignment research center) · redwood research · conjecture · far ai · openai · google deepmind · apollo research · metr · safe superintelligence
Interpretability tools help researchers and engineers understand what is happening inside AI models—why they produce certain outputs and how internal representations form. They are used by AI safety researchers, ML engineers, and enterprises needing to audit or explain model behavior.
neuronpedia · baukit · translucence · goodfire · eleuther ai lm evaluation harness
scale ai red team · hiddenlayer · calypso ai · giskard · inspect (uk aisi) · haize labs
credo ai · holistic ai · fairly ai · nist ai rmf tooling · cot ai · fiddler ai · ibm watsonx.governance · trustible
guardrails ai · lakera · llama guard (meta) · nvidia nemo guardrails · whylabs · arthur ai · straiker · willow · protect ai · robust intelligence · prompt security
anthropic constitutional ai · scale rlhf · predibase · instructlab (ibm)
tenet · witness ai · aim intelligence · invariant labs · capsule
apart research · aisafety info · bluedot impact
Preference data and RLHF labeling covers the collection, annotation, and curation of human preference judgments used to train and fine-tune language models via reinforcement learning from human feedback. ML teams, alignment researchers, and model developers use these services to generate training signal for model behavior.
scale ai · surge ai · mercor · labelbox · turing · invisible technologies · snorkel ai · appen
Model evaluation and red teaming covers tools and services that test AI models for failures, misalignment, and harmful outputs before deployment. ML engineers, safety researchers, and model developers use these to identify vulnerabilities and benchmark performance on adversarial cases.
braintrust · langsmith · arize ai · patronus ai · gray swan ai
Interpretability tooling comprises software and frameworks that help researchers and practitioners understand, visualize, and analyze how neural networks and large language models make decisions. ML engineers, AI safety researchers, and model developers use these tools to inspect activations, trace computation graphs, and identify learned features.
transluce · eleutherai