Runway wants to turn AI video generation into a live stream you control in real time
Runway previews research on real-time AI video generation using its GWM-1 world model to cut wait times and GPU costs.
Runway outlined its research into streaming AI video in real time, building on GWM-1, its first General World Model introduced in December 2025, which generates video frame by frame from camera movements, robot commands, or audio. The approach targets minimal time-to-first-frame and lower GPU costs, and addresses compounding visual errors by training on the model's own outputs; Decart's MirageLSD and DeepMind's Genie 3 take similar approaches. A research preview developed with Nvidia runs on the Vera Rubin platform and targets a first frame in under 100 milliseconds, but Runway has not announced an availability timeline.
- GWM-1, Runway's first General World Model introduced December 2025, builds on Gen-4.5 and generates video frame by frame.
- Each new segment uses all prior frames as context, so small errors can compound into major distortions over time.
- Runway trains models on their own outputs to teach self-correction, similar to Decart's MirageLSD approach.
- A real-time model built with Nvidia runs on Vera Rubin and targets first-frame latency under 100 milliseconds.
- Use cases include interactive media, gaming, education, and robotics simulation; no release timeline was given.
Full article628 words · extracted from the-decoder.com · click to collapse
Runway has shared a look at its research into real-time video generation. Instead of entering a prompt and waiting, users would stream videos as they describe them.
Today's video models work in separate steps. You enter a prompt, wait a few seconds or minutes, and get a finished video. If the result isn't right, you start over. Runway says users repeatedly report losing the most time generating and revising videos and wants to minimize the time to the first frame, then stream video as users prompt it.
Runway first discussed this approach in March with Runway Characters. It uses GWM-1, the company's first "General World Model," which Runway introduced in December 2025. GWM-1 builds on Gen-4.5, generates video frame by frame, and accepts camera movements, robot commands, or audio as controls. Just a few weeks ago, Runway showed Solaris, a system that uses Gen-4.5 to generate user interfaces frame by frame. It responds to clicks or voice input.
Real-time generation could cut waiting and GPU costs
Runway argues that real-time generation closes the gap between an idea and its execution. With instant feedback, users would spend most of their time actively steering the video rather than waiting.
Runway also points to lower costs. Faster models use less GPU time, making them more cost-efficient. According to Runway, the cost per output at a given quality level determines which applications make economic sense. Instant generation would lower that threshold, making previously unprofitable applications viable.
Small visual errors can grow into major distortions
A text model can correct itself mid-sentence, but a video model builds each frame on the previous one, allowing small errors to compound into major distortions over time. Runway describes this as the central problem with LLM-based approaches and addresses it by training the model on its own outputs rather than only error-free inputs, teaching it to correct its own deviations instead of amplifying them.
Startup Decart used a similar approach for its real-time model MirageLSD, deliberately exposing it to flawed or distorted images during training. Google Deepmind says its world model Genie 3 keeps interactive worlds consistent for several minutes at 24 frames per second in 720p.
According to Runway, real-time generation shifts the compute load from training to use. The model must produce each frame fast enough to keep up with playback while running on hardware shared by several sessions at once.
World models would train robots and robotaxis
Runway sees interactive applications as the biggest long-term use case for AI-generated media. Education, gaming, and robotics need video that responds as quickly as the person watching it, the company says. Evaluating how robots or autonomous vehicles act in the world also calls for environments that generate in real time and respond instantly to edge cases. Runway previously introduced GWM Robotics, a variant of GWM-1 that generates synthetic training data for robots.
Waymo is taking a similar approach with the Waymo World Model, which is based on Genie 3 and adapted for road traffic. It lets Waymo simulate situations its fleet has never observed, such as an encounter with an elephant, a tornado, or a flooded residential neighborhood. According to Waymo, the Waymo Driver travels billions of miles in virtual worlds before encountering scenarios on public roads.
In March, Runway also showed a research preview of a real-time model developed with Nvidia at the chipmaker's GTC conference. It runs on the Vera Rubin platform and is designed to deliver the first frame in under 100 milliseconds. Runway hasn't announced a timeline for availability.