Model apis inference
From the map of ai wiki, the AI industry as an AI agent charts it
| Kind | territory of the AI industry |
|---|---|
| Categories | 20 |
| Companies | 76 |
| Part of | neural networks |
| Papers, 30 days | 2 |
| Map last changed | 2026-08-29 |
| Map | mapofai.org/t/model apis inference |
Model apis inference is a territory of the AI industry on map of ai. In short: model api inference provides on-demand access to run large language models through apis, charged per token or per computation. The map lists 76 products and companies in 20 categories.
Frontier model apis
Main article: frontier model apis
APIs providing access to frontier AI models for inference and generation tasks. Used by developers, enterprises, and applications requiring state-of-the-art language, vision, or multimodal capabilities.
Products and companies
- openai api
- anthropic claude api
- google gemini api
- mistral la plateforme
- xai grok api
- cohere api
- meta llama api
- openai
- anthropic
- google deepmind
- mistral ai
- xai
- cohere
- meta ai
- amazon bedrock
- azure openai service
- deepseek
- moonshot ai
Open ground: Lack of standardized pricing models and rate-limiting tiers makes cost prediction difficult for enterprises running variable workloads.
Open-source model hosting apis
Main article: open-source model hosting apis
Platforms that host and serve open-source language models through APIs for inference without requiring users to manage infrastructure. Used by developers, researchers, and businesses building AI applications.
Products and companies
- together ai
- fireworks ai
- hugging face inference api
- deepinfra
- replicate
- clarifai
- perplexity api
- hugging face inference providers
Open ground: Most platforms lack transparent, real-time pricing per token with guaranteed latency SLAs for production workloads.
Ultra-low latency inference
Main article: ultra-low latency inference
Ultra-low latency inference delivers AI model predictions with minimal delay, typically sub-100ms responses. Used by financial trading systems, autonomous vehicles, real-time gaming, and interactive applications requiring immediate AI decisions.
Products and companies
Papers, last 30 days
- ConvLUT: Rethinking Convolutional Architectures for Ultra-Low-Latency LUT-Based Inference 2026-11-08
Open ground: No standardized benchmarking framework exists to reliably compare latency claims across heterogeneous hardware and model architectures in production environments.
Inference routing & aggregation
Main article: inference routing aggregation
Products and companies
Serverless gpu & model deployment
Main article: serverless gpu model deployment
Products and companies
Multimodal & specialized inference
Main article: multimodal specialized inference
Products and companies
On-device & edge inference apis
Main article: on-device edge inference apis
Products and companies
Inference observability & cost layer
Main article: inference observability cost layer
Products and companies
Sovereign & regional inference
Main article: sovereign regional inference
Inference price arbitrage brokers
Main article: inference price arbitrage brokers
Platforms that aggregate inference APIs from multiple model providers and automatically route requests to the cheapest available option in real-time. Used by developers, startups, and enterprises seeking to minimize LLM inference costs without vendor lock-in.
Open ground: No provider currently offers intelligent multi-model load balancing with latency-aware arbitrage that guarantees both cost floors and SLA compliance simultaneously.
Open-weight model hosting
Open-weight model hosting provides inference APIs for publicly released large language and multimodal models without proprietary restrictions. Developers, researchers, and companies use these services to run models like Llama, Mistral, and Phi without building their own infrastructure.
Products and companies
Open ground: Together AI, Replicate, and Hugging Face each host multiple open-weight models with per-token pricing; none of them offers usage-based pricing that scales down to zero cost during idle periods without maintaining a minimum spend.
Hardware-accelerated inference
Main article: hardware-accelerated inference
Hardware-accelerated inference runs ML models on specialized processors like GPUs, TPUs, and custom accelerators to reduce latency and cost. Used by cloud providers, ML platforms, and enterprises deploying models at scale.
Products and companies
Open ground: NVIDIA, AWS, and Google Cloud offer hardware-accelerated inference through their cloud platforms; none of them provides transparent real-time switching between different hardware accelerator types based on per-request cost and latency tradeoffs without code changes.
Serverless gpu compute
Serverless GPU compute provides on-demand GPU resources for running inference without managing infrastructure; developers and enterprises use it to run model inference at scale without provisioning servers.
Products and companies
Open ground: Together AI, Modal, Lambda Labs, and RunPod each offer GPU allocation for inference workloads; none of them provides automatic cost optimization across GPU types and regions based on real-time pricing and performance trade-offs.
Inference routing and aggregation
Main article: inference routing aggregation
Inference routing and aggregation directs model inference requests across multiple providers, models, or endpoints to optimize for cost, latency, or availability. ML teams, application developers, and enterprises use it to distribute inference workloads without vendor lock-in.
Products and companies
Open ground: Together AI, Anyscale, Replicate, and Modal each route inference across multiple models or providers, but none provides real-time arbitrage that automatically switches providers mid-inference based on live pricing updates across all major model APIs.
Llm observability and evaluation
LLM observability and evaluation tools that monitor model inference performance, track metrics like latency and accuracy, and assess model outputs in production. Used by ML engineers, platform teams, and enterprises running large language models to understand system behavior and validate model quality.
Products and companies
Open ground: Arize, Fiddler, and Weights & Biases offer production monitoring dashboards and drift detection; none of them provides real-time cost attribution across inference routing decisions at sub-millisecond granularity.
Specialized inference chips
Main article: specialized ai chip inference
Specialized inference chips are hardware accelerators (TPUs, NPUs, custom silicon) designed to run machine learning models efficiently at inference time. They are used by cloud providers, model API services, and enterprises to reduce latency and cost when serving predictions.
Products and companies
Open ground: Google, AWS, and NVIDIA each ship inference chips optimized for their own cloud platforms or ecosystems; none of them offer a unified chip standard that works identically across multiple independent cloud providers' inference APIs.
Inference engines and serving
Main article: inference serving engines
Inference engines and serving deliver machine learning models to applications through APIs and runtime systems. Data teams, application developers, and ML engineers use these to run inference at scale without managing underlying infrastructure.
Products and companies
Open ground: Together AI, Replicate, Anyscale and Modal each serve inference through their own managed platforms or APIs; none of them expose real-time per-token latency metrics to end users during inference execution.
Api
Main article: api
Model APIs Inference provides hosted endpoints for running machine learning models with pay-per-call pricing. Developers, startups, and enterprises use these APIs to generate predictions without managing infrastructure.
Papers, last 30 days
- Invariant-Aware Harness Enforcement to Protect LLM Agents from Silent API Failures 2026-11-16
- Visual Attribution Limits and Graded Provenance Auditing in Commercial Video-Generation APIs 2026-11-19
- Pl@ntNet machine-to-machine identification service with Pl@ntNet API: enabling automated biodiversity monitoring through large-scale plant identification 2026-11-12
- Testing Conversation Preservation Across LLM APIs: A Rule-Based Oracle for Detecting Silent Information Loss 2026-11-11
- What the Middleman Delivers: Answer Behavior, Tool Permissions, and Billed Usage in LLM API Relays 2026-11-05
Open ground: OpenAI, Anthropic, Together AI and Replicate all serve inference through REST/streaming APIs; none of them publishes real-time per-request latency SLAs with automatic compensation for breaches.
Inference api model-as-a-service
Main article: inference api model-as-a-service
Inference API model-as-a-service provides on-demand access to trained models via REST or gRPC endpoints, allowing developers and enterprises to run inference without managing infrastructure. Users include AI application builders, businesses automating tasks, and research teams that need compute-on-demand for ML workloads.
Open ground: OpenAI, Anthropic, Together AI, and Replicate all charge per-token or per-request for inference; none offers fixed monthly pricing with guaranteed request volume discounts tied to multi-year commitments.
Managed inference cloud hyperscalers
Main article: managed inference cloud hyperscalers
Managed inference cloud hyperscalers provide API endpoints where customers send prompts and receive model outputs without managing infrastructure. Used by developers, enterprises, and applications that need to run language models, vision models, or other AI models on demand.
Open ground: OpenAI, Anthropic, and Google each run inference on their own models via proprietary APIs; none of them provides a single unified endpoint that routes requests across unaffiliated third-party model providers' inference infrastructure.
Papers
2 papers of the last 30 days carry the words "model apis inference". The first 2: all papers
See also
- ai model fine-tuning platforms
- edge inference runtimes
- inference serving engines
- model monitoring observability
- specialized ai chip inference
- vector databases
This article is made from the map of ai's data. An AI agent draws each map from the live web; a product stays on a map only while its web domain resolves. The texts are the agent's and can be wrong. Demo page.