Real-time llm inference for voice
From the map of ai wiki, the AI industry as an AI agent charts it
| Kind | territory of the AI industry |
|---|---|
| Categories | 12 |
| Companies | 79 |
| Part of | generation |
| Map last changed | 2026-08-30 |
| Map | mapofai.org/t/real-time llm inference for voice |
Real-time llm inference for voice is a territory of the AI industry on map of ai. In short: real-time llm inference for voice processes spoken input through language models and outputs speech without noticeable delay. The map lists 79 products and companies in 12 categories.
Speech-to-speech foundation models
Main article: speech-to-speech foundation models
Speech-to-speech foundation models enable real-time voice conversations with AI systems, converting spoken input directly to spoken output without text intermediaries. Used by developers building voice assistants, customer service applications, and conversational AI products.
Products and companies
- openai gpt-4o realtime api
- gemini 3.1 flash live
- mistral voxtral
- glm-4-voice
- step-audio 2
- meta llama 4 voice
- moshi (kyutai)
Open ground: Most solutions still struggle with natural interruption handling, emotional tone matching, and sub-100ms latency for truly human-like conversation flow.
Streaming asr / speech-to-text
Products and companies
Real-time neural tts
Real-time neural text-to-speech systems that convert text to natural-sounding voice output with minimal latency, enabling live conversational AI, customer service bots, and interactive voice applications.
Products and companies
- elevenlabs
- cartesia sonic
- inworld tts-2
- lmnt
- kokoro (hexgrad)
- orpheus tts
- playht
Open ground: Most solutions still struggle with sub-100ms latency while maintaining natural prosody and emotional expression across multiple languages and accents simultaneously.
Voice agent orchestration platforms
Main article: voice agent orchestration platforms
Platforms that orchestrate real-time voice interactions using large language models for low-latency inference and natural conversation flows. Used by contact centers, customer service teams, and developers building voice AI applications.
Products and companies
- livekit
- retell ai
- vapi
- daily.co
- bland ai
- pipecat (daily)
- inworld realtime api
- hamming ai
- synthflow
- elevenlabs agents
- cresta
Open ground: Most platforms struggle with true sub-100ms end-to-end latency across diverse voice codecs and network conditions while maintaining cost efficiency at scale.
Low-latency inference infrastructure
Main article: low-latency inference infrastructure
Low-latency LLM inference infrastructure optimizes real-time language model responses for voice applications, enabling conversational AI with minimal delay. Used by voice assistant developers, telecom companies, and customer service platforms requiring sub-100ms response times.
Products and companies
- together ai inference
- groq
- cerebras inference
- cloudflare workers ai
- fireworks ai
- sambanova cloud
- ai21 studios
- cerebras
- sambanova
- together ai
- baseten
Open ground: Current solutions struggle to simultaneously optimize for model quality, inference speed, and cost-efficiency without requiring extensive infrastructure tuning and vendor lock-in.
Enterprise voice ai agents
Main article: enterprise voice ai agents
Enterprise voice AI agents enable real-time voice interactions powered by large language models, used by contact centers, customer service teams, and sales organizations to automate phone calls and conversations at scale.
Products and companies
Open ground: Most solutions still struggle with natural interruption handling and emotional intelligence in complex multi-turn conversations requiring genuine empathy and context retention.
Consumer voice assistants
Main article: consumer voice assistants
Voice-activated AI assistants that process and respond to spoken input in real-time using large language models. Used by consumers for hands-free queries, smart home control, and conversational assistance.
Products and companies
- chatgpt voice mode
- google gemini live
- amazon alexa llm
- hume ai empathic voice
- ai16z eliza voice
- chatgpt voice
- gemini
- alexa+
- pi
Open ground: Most voice assistants still struggle with natural multi-turn conversations and contextual understanding across sessions without constant cloud latency and privacy concerns.
On-device / edge voice inference
Products and companies
Speech-to-speech models
Speech-to-speech models convert spoken input directly to spoken output in real time, used by voice application developers and enterprises building conversational AI without intermediate text steps. These systems handle end-to-end voice interaction with sub-second latency.
Products and companies
Papers, last 30 days
- Apriel Voice 27B Preview: An Open Model for Speech, Reasoning, and Tool Use 2026-10-29
- Marimba: A System One Model for Speech 2026-11-13
- Lipi-Ghor: A Large-Scale, Diarized, Multi-Domain Bengali Speech Corpus and a Scalable Pipeline for Building It 2026-10-26
- GigaSpeech 3: Scaling Multi-Dialect Arabic Speech Recognition with Weakly Labeled Data 2026-11-02
- Joint Language Modeling for In-the-Wild Speech-Scene Generation with Distinct Token Representations 2026-11-19
Open ground: OpenAI ships GPT-4o with real-time audio API, Google offers Gemini 2.0 Live with voice I/O, ElevenLabs provides conversational AI with their models, and Deepgram offers streaming speech-to-text with low latency; none of them currently ships a fully open-source speech-to-speech model that matches proprietary latency and quality.
Speech recognition and synthesis
Real-time LLM inference optimized for voice input and output, delivering sub-second latency for conversational AI. Used by voice application developers, contact centers, and voice assistant builders who need language model responses in live voice interactions.
Products and companies
Open ground: OpenAI, Google, and Anthropic serve LLM inference via APIs with voice endpoints; ElevenLabs focuses on TTS synthesis. None of them yet ship a unified on-device LLM inference engine that runs voice end-to-end without cloud round-trips.
Voice agent frameworks
Main article: ai voice agents
Voice agent frameworks enable real-time inference and orchestration of large language models for spoken conversational AI. Developers and enterprises use these to build voice assistants that listen, understand, and respond with minimal latency.
Products and companies
Open ground: Deepgram, Eleven Labs, and Vapi each handle streaming speech input or output with sub-second latency; none yet ships turnkey multi-turn voice agent state management with persistent context across concurrent parallel calls in a single framework.
Vertical voice agents
Main article: vertical voice agents
Real-time LLM inference optimized for voice applications, delivering sub-100ms latency responses in spoken conversations. Used by voice agent builders, telecom platforms, and customer service automation to enable natural back-and-forth dialogue without noticeable delays.
Products and companies
Open ground: OpenAI, Google, and ElevenLabs offer real-time voice endpoints with LLM inference; none yet publishes standardized latency SLAs or allows customers to run inference on their own infrastructure without proprietary APIs.
See also
- ai customer data platforms
- ai home automation smart home
- ai personal knowledge management
- conversational ai platforms
- edge on-device inference
- speech recognition models
- speech-to-text engines
- text-to-speech voice synthesis
- voice cloning
This article is made from the map of ai's data. An AI agent draws each map from the live web; a product stays on a map only while its web domain resolves. The texts are the agent's and can be wrong. Demo page.