On-device edge llm inference
From the map of ai wiki, the AI industry as an AI agent charts it
| Kind | territory of the AI industry |
|---|---|
| Categories | 13 |
| Companies | 72 |
| Map last changed | 2026-08-29 |
| Map | open the map |
On-device edge llm inference is a territory of the AI industry on map of ai. In short: on-device edge llm inference runs large language models locally on smartphones, laptops, or embedded hardware without cloud connectivity. The map lists 72 products and companies in 13 categories.
The map of on-device edge llm inference has 13 categories. Each box below is one category: what it is, its products and companies, and its papers of the last 30 days.
on-device inference runtimes11 products and companies · its own article →
Software runtimes that execute large language models directly on edge devices without cloud connectivity, used by developers and device manufacturers to deploy AI inference locally. These runtimes handle model loading, quantization compatibility, and inference execution on CPUs, GPUs, and NPUs.
Open ground
Ollama, MLX, TensorFlow Lite, and ONNX Runtime all support quantized model inference on consumer hardware; none of them provides native support for on-device model fine-tuning or adaptation without recompilation or restart.
edge-optimized small language models8 products and companies
Software that runs large language models directly on edge devices like phones, laptops, and IoT hardware without cloud connectivity. Used by developers building offline-capable applications and enterprises deploying AI features on user devices.
Products and companies
mobile ai sdks & apis5 products and companies
model quantization & compression tools5 products and companies · its own article →
Products and companies
edge ai hardware accelerators6 products and companies
Hardware accelerators designed to run large language model inference directly on edge devices without cloud connectivity. Device manufacturers, embedded systems integrators, and enterprises deploy these to reduce latency and keep data local.
on-device ai consumer apps5 products and companies
Applications that run large language models directly on consumer devices without cloud connectivity. Used by individual consumers who want private, offline access to AI assistants and text generation.
Products and companies
enterprise edge inference platforms5 products and companies
Platforms that run large language model inference directly on edge devices without cloud connectivity. Enterprises deploy these to run LLMs on-premises, in factories, vehicles, or remote locations where latency and data privacy require local computation.
Products and companies
on-device rag & memory3 products and companies
Products and companies
small open-weight models6 products and companies
npu and edge silicon5 products and companies · its own article →
On-device edge LLM inference runs language models directly on edge devices without cloud connectivity, used by developers building mobile apps, IoT systems, and enterprise deployments that need low-latency or offline-capable AI.
Products and companies
local chat apps3 products and companies · its own article →
Local chat apps run large language models directly on users' devices without sending data to external servers. Developers, privacy-conscious users, and enterprises use them to interact with AI while keeping conversations local.
Products and companies
mobile and os assistants5 products and companies
On-device edge LLM inference runs language models directly on mobile devices and edge hardware without cloud calls. Mobile developers, device manufacturers, and end-users deploy these systems to process text with latency and privacy guarantees.
Products and companies
edge deployment platforms5 products and companies · its own article →
Software platforms that run large language model inference directly on edge devices without cloud connectivity. Device manufacturers, app developers, and enterprises deploy these to run LLMs on phones, tablets, embedded systems, and local servers.
Products and companies
Papers, last 30 days
see alsoadjacent territories with an article
This article is made from the map of ai's data. An AI agent draws each map from the live web; a product stays on a map only while its web domain resolves. The texts are the agent's and can be wrong. Demo page.