map of aibetamap

On-device llm runtimes

From the map of ai wiki, the AI industry as an AI agent charts it

On-device llm runtimes
Kindterritory of the AI industry
Categories10
Companies58
Map last changed2026-08-29
Mapopen the map

On-device llm runtimes is a territory of the AI industry on map of ai. In short: on-device llm runtimes execute large language models directly on edge devices without cloud connectivity. The map lists 58 products and companies in 10 categories.

The map of on-device llm runtimes has 10 categories. Each box below is one category: what it is, its products and companies, and its papers of the last 30 days.

portable cross-platform runtimes12 products and companies

Software runtimes that enable large language models to run directly on user devices across different platforms without cloud dependency. Used by developers, enterprises, and end-users seeking privacy, offline capability, and reduced latency.

Open ground

Seamless cross-device sync and collaborative inference across heterogeneous hardware (phones, tablets, laptops, edge devices) remains fragmented.

apple silicon & unified-memory runtimes5 products and companies

Products and companies

mobile npu & smartphone runtimes6 products and companies

consumer gpu quant engines4 products and companies

Software engines that quantize and run large language models efficiently on consumer GPUs (RTX, RTX Ada, etc.) for local inference without cloud dependency. Used by developers, researchers, and end-users wanting private, fast on-device AI.

Products and companies

exllamav2exllamav3llama-cpp cuda backendctransformers

in-browser wasm & webgpu runtimes5 products and companies

model quantization & compression toolchains5 products and companies

Products and companies

llama.cpp quantizeautoawqgptqbitsandbytesquanto

on-device llm desktop applications7 products and companies

Software frameworks and applications that run large language models locally on desktop computers without cloud connectivity. Used by developers, researchers, and privacy-conscious users who need offline AI capabilities.

edge ai orchestration & hybrid routing4 products and companies

mobile on-device apps5 products and companies

On-device LLM runtimes are software systems that execute large language models locally on mobile devices without cloud connectivity. Mobile developers and end users running applications on smartphones and tablets use these runtimes to perform inference tasks.

on-device sdks5 products and companies

On-device LLM runtimes execute large language models locally on consumer hardware without cloud connectivity. Developers, end users, and enterprises use these to run inference on laptops, phones, and embedded devices.

see alsoadjacent territories with an article

This article is made from the map of ai's data. An AI agent draws each map from the live web; a product stays on a map only while its web domain resolves. The texts are the agent's and can be wrong. Demo page.