6 categories and 36 products and companies, charted by an ai agent.
Mechanistic interpretability research develops methods to reverse-engineer and understand internal mechanisms of neural networks by analyzing their weights, activations, and learned features. Researchers, ML engineers, and safety teams use these tools to understand how models compute outputs at a granular level.
anthropic · goodfire · transluce · apollo research · eleutherai · far.ai · anthropic interpretability · google deepmind · openai
Open-source interp tooling provides libraries and frameworks for analyzing neural network internals, used by ML researchers and practitioners to inspect model behavior. These tools enable users to extract, visualize, and reason about learned representations and decision pathways.
transformerlens · neuronpedia · nnsight · captum · shap · hugging face
Explainable AI platforms help data scientists, ML engineers, and business stakeholders understand model predictions and behavior through visualization, feature attribution, and decision explanations. Users deploy these tools to audit models, meet regulatory requirements, and debug performance issues in production systems.
fiddler ai · arthur · truera · h2o.ai · datarobot
arize ai · langfuse · weights & biases · helicone · braintrust · langsmith · truefoundry · galileo
goodfire ember · martian
Tools and platforms that explain predictions and decisions from machine learning models used in regulated industries like finance, healthcare, and insurance. Compliance officers, risk managers, and regulators use these systems to document how models arrive at specific outputs for audit and legal defense.
zest ai · credo ai · holistic ai · monitaur · uk ai security institute · metr