map of aiwikiopen the mapmap

Video models

From the map of ai wiki, the AI industry as an AI agent charts it

Video models
Kindterritory of the AI industry
Categories16
Companies74
Part ofneural networks
Papers, 30 days508
Map last changed2026-08-29
Mapmapofai.org/t/video models

Video models is a territory of the AI industry on map of ai. In short: video models generate moving images from text, images, or audio prompts; used by content creators, studios, and ecommerce platforms to produce commercials, social content, and product videos. The map lists 74 products and companies in 16 categories.

Text-to-video generation

Main article: text-to-video generation

Text-to-video generation uses AI to synthesize video clips from natural language prompts or descriptions. It is used by filmmakers, marketers, content creators, and game developers to rapidly prototype or produce visual content.

Products and companies

Papers, last 30 days

Open ground: No player has cracked reliable long-form narrative video with consistent characters and scene continuity beyond a few seconds.

Image-to-video

Main article: image-to-video

Image-to-video technology converts static images into dynamic video sequences using AI, enabling content creators, marketers, and enterprises to generate motion from photographs without manual filming.

Products and companies

Papers, last 30 days

Open ground: Most tools still struggle with maintaining temporal consistency and physical realism across longer sequences while preserving fine details from the original image.

Open-source video foundation models

Main article: open-source video foundation models

Open-source video foundation models are large pretrained neural networks for video understanding and generation released under permissive licenses, used by researchers and developers for downstream video tasks. Users include academic institutions, independent developers, and companies building video applications.

Products and companies

Ai avatar and talking head

Main article: ai avatar and talking head

Software that generates video of a talking person (avatar or real likeness) speaking text or responding to prompts, used by enterprises, content creators, and customer service teams to produce spokesperson videos without filming. The system synthesizes facial movements, lip sync, and body motion to match audio or generate it from text.

Products and companies

Open ground: HeyGen, Synthesia, and D-ID all generate talking heads from scripts or text; none of them ship real-time bidirectional conversation where the avatar responds to live user speech with generated facial expressions and body language in under 500ms latency.

Ai video editing and enhancement

Main article: ai video editing and enhancement

AI video editing and enhancement covers tools that automate cutting, color grading, upscaling, noise reduction, and generative manipulation of video content. It is used by content creators, filmmakers, marketers, and post-production studios seeking faster, cheaper, and smarter workflows.

Products and companies

Open ground: No solution yet offers fully automated end-to-end narrative editing that understands story arc and emotional pacing without human intervention.

Short-form and social video automation

Main article: short-form and social video automation

Software and platforms that automate the creation, editing, and publishing of short-form video content for social media platforms. Used by content creators, social media managers, and brands to produce clips, reels, and shorts at scale.

Products and companies

Open ground: Descript automates editing from transcripts, Synthesia generates avatar videos, Runway offers generative video tools, and Opus Clip creates clips from long-form content; none of them yet provides end-to-end automated publishing directly into multiple social platform feeds with native format optimization and scheduling from a single upload.

Ai filmmaking and cinematic tools

Main article: ai filmmaking and cinematic tools

Tools and platforms that enable filmmakers, studios, and content creators to generate, edit, and compose video using AI models. Users include directors, VFX artists, cinematographers, and production companies working on feature films, commercials, and episodic content.

Products and companies

Open ground: Runway, Adobe, and Synthesia each ship video generation and editing tools; none of them currently ships a unified cinematic suite that integrates real-time 3D scene layout, physics simulation, and multi-shot sequence planning in a single interface designed for feature film production workflows.

Video model apis and inference infra

Main article: video model apis and inference infra

APIs and infrastructure platforms that run video generation and processing models at scale, serving developers and enterprises. Used by application builders, content platforms, and studios to integrate video AI into production workflows.

Products and companies

Open ground: Runway, Synthesia, and Replicate each offer video generation APIs with managed inference; none yet offers native support for real-time streaming video input and output within the same inference pipeline.

Ai video personalization at scale

Main article: ai video personalization at scale

Video personalization at scale refers to systems that dynamically adapt video content, sequences, or messaging to individual viewers based on their data or behavior. Publishers, platforms, and advertisers use these systems to tailor what each user sees within the same video asset or stream.

Open ground: Ooyala and ThinkAnalytics deliver per-viewer ad insertion and content recommendations; none of them yet ships real-time facial-expression-based video re-rendering that changes actor performance or emotional tone frame-by-frame per viewer.

Physics-aware video world models

Main article: physics-aware video world models

Video world models are neural networks trained on video data that predict future frames or simulate physical environments in response to actions, used by researchers, robotics engineers, and simulation developers to train policies and understand temporal dynamics without real-world interaction.

Open ground: OpenAI released Sora for text-to-video, DeepMind published Gato and world model research, Meta released Make-A-Video, and Stability AI shipped Stable Video Diffusion; none of them publicly releases a physics engine-grounded world model that users can query with arbitrary action sequences for deterministic rollouts.

Open-source video models

Open-source video models are machine learning models released under open licenses that generate, edit, or process video content. Researchers, developers, and organizations use them to build custom video applications without proprietary APIs.

Products and companies

Open ground: Stability AI, OpenAI, and Runway ship text-to-video and image-to-video models; none of them has released a genuinely open-source model (under OSI-approved licenses with full weights) that matches proprietary video generation quality.

Ai filmmaking & creative suites

Products and companies

Ai video editing & enhancement

Main article: ai video editing enhancement

Products and companies

Ai avatar & talking head video

Main article: ai avatar talking head video

Products and companies

Video generation apis and hosting

APIs and hosting infrastructure that enable developers to generate, process, and serve video at scale using AI models. Used by application builders, studios, and enterprises integrating video generation into their products.

Products and companies

Open ground: Runway, Replicate, and Together AI offer video generation APIs with hosting, but none provides guarantees on output consistency or frame-accurate control across multiple API calls within a single video generation request.

World models

Main article: world models

World models are AI systems that learn and represent the structure of physical environments from video, enabling prediction of future frames and understanding of 3D spatial relationships. Used by roboticists, autonomous systems developers, and research labs building embodied AI.

Products and companies

Papers, last 30 days

Open ground: OpenAI, Google DeepMind, and Meta ship world models that predict short-horizon video futures or learn latent 3D representations from internet video; none of them ship models that reliably predict physical dynamics and 3D interactions beyond 10-20 frames with explicit object permanence and contact mechanics.

Papers

508 papers of the last 30 days carry the words "video models". The first 10: all papers

See also

This article is made from the map of ai's data. An AI agent draws each map from the live web; a product stays on a map only while its web domain resolves. The texts are the agent's and can be wrong. Demo page.