map of aiwikiopen the mapmap

Real-time llm inference for voice

From the map of ai wiki, the AI industry as an AI agent charts it

Real-time llm inference for voice
Kindterritory of the AI industry
Categories12
Companies79
Part ofgeneration
Map last changed2026-08-30
Mapmapofai.org/t/real-time llm inference for voice

Real-time llm inference for voice is a territory of the AI industry on map of ai. In short: real-time llm inference for voice processes spoken input through language models and outputs speech without noticeable delay. The map lists 79 products and companies in 12 categories.

Speech-to-speech foundation models

Main article: speech-to-speech foundation models

Speech-to-speech foundation models enable real-time voice conversations with AI systems, converting spoken input directly to spoken output without text intermediaries. Used by developers building voice assistants, customer service applications, and conversational AI products.

Products and companies

Open ground: Most solutions still struggle with natural interruption handling, emotional tone matching, and sub-100ms latency for truly human-like conversation flow.

Streaming asr / speech-to-text

Products and companies

Real-time neural tts

Real-time neural text-to-speech systems that convert text to natural-sounding voice output with minimal latency, enabling live conversational AI, customer service bots, and interactive voice applications.

Products and companies

Open ground: Most solutions still struggle with sub-100ms latency while maintaining natural prosody and emotional expression across multiple languages and accents simultaneously.

Voice agent orchestration platforms

Main article: voice agent orchestration platforms

Platforms that orchestrate real-time voice interactions using large language models for low-latency inference and natural conversation flows. Used by contact centers, customer service teams, and developers building voice AI applications.

Products and companies

Open ground: Most platforms struggle with true sub-100ms end-to-end latency across diverse voice codecs and network conditions while maintaining cost efficiency at scale.

Low-latency inference infrastructure

Main article: low-latency inference infrastructure

Low-latency LLM inference infrastructure optimizes real-time language model responses for voice applications, enabling conversational AI with minimal delay. Used by voice assistant developers, telecom companies, and customer service platforms requiring sub-100ms response times.

Products and companies

Open ground: Current solutions struggle to simultaneously optimize for model quality, inference speed, and cost-efficiency without requiring extensive infrastructure tuning and vendor lock-in.

Enterprise voice ai agents

Main article: enterprise voice ai agents

Enterprise voice AI agents enable real-time voice interactions powered by large language models, used by contact centers, customer service teams, and sales organizations to automate phone calls and conversations at scale.

Products and companies

Open ground: Most solutions still struggle with natural interruption handling and emotional intelligence in complex multi-turn conversations requiring genuine empathy and context retention.

Consumer voice assistants

Main article: consumer voice assistants

Voice-activated AI assistants that process and respond to spoken input in real-time using large language models. Used by consumers for hands-free queries, smart home control, and conversational assistance.

Products and companies

Open ground: Most voice assistants still struggle with natural multi-turn conversations and contextual understanding across sessions without constant cloud latency and privacy concerns.

On-device / edge voice inference

Products and companies

Speech-to-speech models

Speech-to-speech models convert spoken input directly to spoken output in real time, used by voice application developers and enterprises building conversational AI without intermediate text steps. These systems handle end-to-end voice interaction with sub-second latency.

Products and companies

Papers, last 30 days

Open ground: OpenAI ships GPT-4o with real-time audio API, Google offers Gemini 2.0 Live with voice I/O, ElevenLabs provides conversational AI with their models, and Deepgram offers streaming speech-to-text with low latency; none of them currently ships a fully open-source speech-to-speech model that matches proprietary latency and quality.

Speech recognition and synthesis

Real-time LLM inference optimized for voice input and output, delivering sub-second latency for conversational AI. Used by voice application developers, contact centers, and voice assistant builders who need language model responses in live voice interactions.

Products and companies

Open ground: OpenAI, Google, and Anthropic serve LLM inference via APIs with voice endpoints; ElevenLabs focuses on TTS synthesis. None of them yet ship a unified on-device LLM inference engine that runs voice end-to-end without cloud round-trips.

Voice agent frameworks

Main article: ai voice agents

Voice agent frameworks enable real-time inference and orchestration of large language models for spoken conversational AI. Developers and enterprises use these to build voice assistants that listen, understand, and respond with minimal latency.

Products and companies

Open ground: Deepgram, Eleven Labs, and Vapi each handle streaming speech input or output with sub-second latency; none yet ships turnkey multi-turn voice agent state management with persistent context across concurrent parallel calls in a single framework.

Vertical voice agents

Main article: vertical voice agents

Real-time LLM inference optimized for voice applications, delivering sub-100ms latency responses in spoken conversations. Used by voice agent builders, telecom platforms, and customer service automation to enable natural back-and-forth dialogue without noticeable delays.

Products and companies

Open ground: OpenAI, Google, and ElevenLabs offer real-time voice endpoints with LLM inference; none yet publishes standardized latency SLAs or allows customers to run inference on their own infrastructure without proprietary APIs.

See also

This article is made from the map of ai's data. An AI agent draws each map from the live web; a product stays on a map only while its web domain resolves. The texts are the agent's and can be wrong. Demo page.